Game apparatus, storage medium, and computer program that adjust level of game difficulty

ABSTRACT

A game apparatus, a storage medium and a computer program, suitable to games that can be experienced by a player using an entire body to play a game. The game apparatus comprises: a storage section for storing a predetermined assigned movement for a player; an assigned movement display section for displaying the assigned movement, thereon; a movement detection section for detecting a movement of the player; a player movement display section for displaying the movement of the player detected by the movement detection section, thereon; a similarity decision section for comparing the assigned movement with the movement of the player detected by the movement detection section, to decide a similarity between the assigned movement and the movement of the player; and a game operation section for processing a score on the basis of the similarity decided by the similarity decision section.

This is a Divisional of application Ser. No. 09/784,315 filed Feb. 16,2001 now U.S. Pat. No. 6,663,491. The entire disclosure of the priorapplication is hereby incorporated by reference herein in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a game apparatus that detects amovement of a player and then operates a game according to the movementdetected, and so on.

2. Description of Related Art

Various types of a game apparatus today typically has an input deviceused by a player to enter various control commands, and the input devicehas an operating unit manipulated by the player to play the game. Theoperating unit of the input device of the game apparatus is typically ajoy-hand, a joystick or similar device. When the player manipulates theoperating unit while playing a game, the input device sends a signalindicative of the player's action to the game system. The game apparatusthen performs a particular game process, such as moving or otherwiseoperating the object in the game image affected by the player's action.A drawback to game apparatuses having such a conventional input deviceis that while the player uses the player's hands and feet to manipulatethe operating unit and play the game, it is difficult for the player toenjoy the experience of using the entire body to play the game.

Video game apparatuses having a CCD camera or other imaging device areknown from the literature as one attempt to solve this problem. Thistype of video game apparatus uses the imaging device to image theplayer. A feature extraction process then removes the background fromthe captured image to extract an image of just the player. The shape(body shape) and actions of the player are then recognized using theresult of this feature extraction process.

Unexamined Published Japanese Patent Application No.hei-7-155467, forexample, teaches a video game apparatus of this type. The video gameapparatus described in Unexamined Published Japanese Patent ApplicationNo.hei-7-155467 uses a CCD camera to capture an image of the player infront of a screen. The captured image is then converted to a digitalimage signal to which a feature extraction process is applied to removethe background and extract only the player. More specifically, colordifferences between the screen color and the color of the player in thecaptured image are used to extract an image of just the player. Thisoperation focuses on the screen color (which is blue), and uses a blueextraction circuit to remove the screen (that is, the background image).

This video game apparatus displays an instruction pattern having whiteor red areas near the four corners of the screen. The player holds awhite or red flag in each hand and appropriately raises and lowers theflags according to the pattern displayed on the screen. The gameapparatus detects whether the color of the flag raised or lowered by theplayer matches the color displayed on the screen.

The video game apparatus according to an earlier development asdescribed above, has the following technical problem. Since the playeris photographed by the CCD camera, more specifically, the characteristicextracting process or the like for detecting the movement of the playertakes such a long operation time as to make it difficult to establishthe processing rate desired for the execution of the game. Therefore,the recognition (the pattern recognition) of the flag raised and loweredby the player in the video game apparatus as disclosed in UnexaminedPublished Japanese Patent Application No.hei-7-155467, is decideddepending on whether or not the color data of the flag exceed apredetermined amount (or area) at a predetermined region. Accordingly, amatch or mismatch is thus determined using the flag color, so that thevideo game apparatus does not detect the action itself of the player.

As one game apparatus for the player to enjoy the game physically withthe whole body, there is known a game apparatus for a dance game. Thisgame apparatus comprises an input unit having a plurality of switches tobe turned ON/OFF by the weight or the like of the player, and the playerplays the game on that input unit. Specifically, the game apparatus forplaying this dance game is constructed such that the game can beprogressed while calculating the score by detecting whether or not theplayer depresses (or rides on) the switches at a given timing.Therefore, the game apparatus does not detect and decide the danceitself of the player, but the dance game is realized and held bydetecting the input timing to the input unit.

However, a problem with the above video game apparatus taught inUnexamined Published Japanese Patent Application No.hei-7-155467 is thatan extremely complex image process is required to capture an image ofplayer's movement and identify player's movement within the image. Inaddition, player's movement is detected by a pattern recognition processbased on the color of flags held in the player's hands. It is thereforenot possible to play the game if the player is not holding flags of thecorrect colors, and players cannot be said to enjoy experiencing playingthe game with the entire body. The result, therefore, is that the gameis played with only hand's movements detected using flags, whichfunction as the above-noted operating unit.

Furthermore, the game is played by the player simply raising andlowering red or white flags held in the hands to match a particularpattern having red or white areas displayed in the four corners of thescreen. The game is thus extremely simple.

Further, as described above, either the video game apparatus, asdisclosed in Unexamined Published Japanese Patent ApplicationNo.hei-7-155467, or the game apparatus for the dance game according toan earlier development progresses the game by the simple movement ofraising the flag in the given region or by the simple input ofdepressing the given switch. Therefore, the game contents aremonotonous, and the inputting action is specified to a predeterminedmovement to depress the switch so that the game apparatus is troubled bythe following problem. It is entrusted on the player whether or not theplayer plays the game by using the entire body. In other words, the gameapparatus only decides whether or not the switch is depressed.Therefore, the movement on the player side has no relation to theprogress or score of the game so that it results in the so-called“waste”.

Furthermore, according to the video game apparatus as disclosed inUnexamined Published Japanese Patent Application No.hei-7-155467, sincethe player would be indicated with an answer upon the display of theinstruction patterns on the four corners of the screen, the apparatus isincapable of giving clues to allow the player to guess a correctmovement or providing incorrect clues to confuse the player.Accordingly, the only factor reflected to the game result is thequickness of the player's movement, rendering the game quite simple, sothat the player may quickly loose his interest in playing the game.

SUMMARY OF THE INVENTION

The present invention was developed in view of the above-describedproblems.

An object of the present invention is to provide a game apparatus and soon, suitable to games that can be experienced by a player using anentire body to play a game.

Another object of the present invention is to provide a game apparatusand so on, which can progress a game and process a score according to amovement using an entire body without restricting an inputting action.

A further object of the present invention is to provide a game apparatusand so on, which can realize a body sensing game requiring a player touse an entire body and vary contents of the game by outputting hints.

In accordance with the first aspect of the present invention, a gameapparatus (for example, a game apparatus 1700 shown in FIG. 1)comprises: a storage section (for example, a data storage medium 1500shown in FIG. 8) for storing a predetermined assigned movement for aplayer; an assigned movement display section (for example, a gameoperation unit 1010, a game image generation unit 1032, and a datastorage medium 1500 shown in FIG.) for displaying the assigned movement,thereon; a movement detection section (for example, a movement detectionunit 1200 shown in FIG. 8) for detecting a movement of the player; aplayer movement display section (for example, a game operation unit1010, a player image generation unit 1034, and a data storage medium1500 shown in FIG. 8) for displaying the movement of the player detectedby the movement detection section, thereon; a similarity decisionsection (for example, a similarity decision unit 1016, and an assignmentdata 1530 shown in FIG. 8) for comparing the assigned movement with themovement of the player detected by the movement detection section, todecide a similarity between the assigned movement and the movement ofthe player; and a game operation section (for example, a game operationunit 1010, and a data storage medium 1500 shown in FIG. 8) forprocessing a score on the basis of the similarity decided by thesimilarity decision section.

In accordance with the second aspect of the present invention, accordingto a storage medium having a computer-executable program recordedthereon, the computer comprising a movement detection section fordetecting a movement of a player, the program comprises: a program codeof executing a predetermined game (for example, a game program 1510shown in FIG. 8); a program code of storing a predetermined assignedmovement (for example, an assignment data 1530 shown in FIG. 8); aprogram code of displaying the assigned movement (for example, a gameprogram 1510 shown in FIG. 8); a program code of inputting an input databased on a movement of the player detected by the movement detectionsection (for example, a movement input program 1040 shown in FIG. 8); aprogram code of displaying the movement of the player on the basis ofthe input data inputted (for example, a game program 1510 shown in FIG.8); a program code of comparing the assigned movement with the inputdata, to decide a similarity between the assigned movement and the inputdata (for example, a game program 1510 and a decision reference data1536 shown in FIG. 8); and a program code of processing a score on thebasis of similarity decided (for example, a game program 1510 shown inFIG. 8).

In accordance with the third aspect of the present invention, a computerprogram comprises program code means for performing the steps of:storing a predetermined assigned movement for a player; displaying theassigned movement; detecting a movement of the player; displaying themovement of the player detected; comparing the assigned movement withthe movement of the player detected, to decide a similarity between theassigned movement and the movement of the player; and processing a scoreon the basis of the similarity decided.

According to the game apparatus, the storage medium and the computerprogram in accordance with the first, second and third aspects of thepresent invention, the player performs the assigned movement displayedso as to match the assigned movement, and thereby the game is advanced.

Thereafter, the movement of the player is detected and compared with theassigned movement, to decide the similarity. Based on the result ofdeciding the similarity, the score is processed, and the game progressesaccording to the score. For example, the player can enjoy comparing theplayer's movement with the assigned movement by displaying the result ofthe processing the score. Accordingly, because the player moves so as tomatch the assigned dance, the player can enjoy the experience of playingthe game using the entire body.

Further, the player can play the game while checking the player's ownmovement detected and displayed. Accordingly, the player can enjoywatching the player's own movement.

Furthermore, the game apparatus in accordance with the first aspect ofthe present invention can detect the player's movement without requiringa complicated image processing operation because the movement detectionsection detects the player's movement. That is, the game apparatus isconstructed so as to be different from a conventional flag-raising typevideo game apparatus that detects the player's movement based on theflags held in the player's hands, and directly detects the movement ofthe player's entire body. Accordingly, the player can enjoy theexperience of playing the game with the entire body.

It should be noted that the “score” as used herein refers notnecessarily to a strict numerical point system, but to an objectivedecision corresponding to the detected degree of similarity between theplayer's movement and the assigned movement.

In accordance with the fourth aspect of the present invention, a gameapparatus (for example, a game apparatus 1700 shown in FIG. 1)comprises: a storage section (for example, a data storage medium 1500shown in FIG. 8) for storing a predetermined assigned movement for aplayer; an assigned movement display section (for example, a gameoperation unit 1010, a game image generation unit 1032, and a datastorage medium 1500 shown in FIG. 8) for displaying the assignedmovement, thereon; a sound generation section (for example, a gameoperation unit 1010, a sound generation unit 1040, and a data storagemedium 1500 shown in FIG. 8) for generating at least one of a sound, amusic and a sound effect; a movement detection section (for example, amovement detection unit 1200 shown in FIG. 8) for detecting a movementof the player; a similarity decision section (for example, a similaritydecision unit 1016, and a data storage medium 1500 shown in FIG. 8) forcomparing the assigned movement with the movement of the player detectedby the movement detection section, to decide a similarity between theassigned movement and the movement of the player; and a game operationsection (for example, a game operation unit 1010, and a data storagemedium 1500 shown in FIG. 8) for processing a score on the basis of thesimilarity decided by the similarity decision section.

In accordance with the fifth aspect of the present invention, accordingto a storage medium having a computer-executable program recordedthereon, the computer comprising a movement detection section fordetecting a movement of a player, the program comprises: a program codeof executing a predetermined game (for example, a game program 1510shown in FIG. 8); a program code of storing a predetermined assignedmovement (for example, an assignment data 1530 shown in FIG. 8); aprogram code of displaying the assigned movement (for example, a gameprogram 1510 shown in FIG. 8); a program code of inputting an input databased on a movement of the player detected by the movement detectionsection (for example, a movement input program 1540 shown in FIG. 8); aprogram code of generating at least one of a sound, a music and a soundeffect (for example, a game program 1510, and a sound data 1532 shown inFIG. 8); a program code of comparing the assigned movement with theinput data, to decide a similarity between the assigned movement and theinput data (for example, a game program 1510, and a decision referencedata 1536 shown in FIG. 8); and a program code of processing a score onthe basis of similarity decided (for example, a game program 1510 shownin FIG. 8).

In accordance with the sixth aspect of the present invention, a computerprogram comprises program code means for performing the steps of:storing a predetermined assigned movement for a player; displaying theassigned movement; generating at least one of a sound, a music and asound effect; detecting a movement of the player; comparing the assignedmovement with the movement of the player detected, to decide asimilarity between the assigned movement and the movement of the player;and processing a score on the basis of the similarity decided.

According to the game apparatus, the storage medium and the computerprogram in accordance with the fourth, fifth and sixth aspects of thepresent invention, the player performs the assigned movement displayedso as to match the assigned movement, and thereby the game is advanced.

Thereafter, the movement of the player is detected and compared with theassigned movement, to decide the similarity. Based on the result ofdeciding the similarity, the score is processed, and the game progressesaccording to the score. For example, the player can enjoy comparing theplayer's movement with the assigned movement by displaying the result ofthe processing the score. Accordingly, because the player moves so as tomatch the assigned dance, the player can enjoy the experience of playingthe game using the entire body.

Further, the player can enjoy the game while listening to at least oneof the sound, the music and the sound effect generated. And further, theplayer can move to the rhythm or the tempo of the sound or the music, sothat the player can rhythmically move. Furthermore, the player movesrhythmically to the sound, the music or the sound effect, thereby otherpeople can enjoy watching the player's movement and also be motivated toplay the game themselves.

Furthermore, the game apparatus in accordance with the fourth aspect ofthe present invention can detect the player's movement without requiringa complicated image processing operation because the movement detectionsection detects the player's movement. That is, the game apparatus isconstructed so as to be different from a conventional flag-raising typevideo game apparatus that detects the player's movement based on theflags held in the player's hands, and directly detects the movement ofthe player's entire body. Accordingly, the player can enjoy theexperience of playing the game with the entire body.

It should be noted that the “sound” as used herein refers is not limitedto the peoples' voices, but could be bird songs or other animal noises.

In accordance with the seventh aspect of the present invention, a gameapparatus (for example, a game apparatus 1700 shown in FIG. 8)comprises: a storage section (for example, a data storage medium 1500shown in FIG. 8) for storing a predetermined assigned movement for aplayer; an assigned movement display section (for example, a gameoperation unit 1010, a game image generation unit 1032, and a datastorage medium 1500 shown in FIG. 8) for displaying the assignedmovement, thereon; a sound generation section (for example, a gameoperation unit 1010, a sound generation unit 1040, and a data storagemedium 1500 shown in FIG. 8) for generating at least one of a sound, amusic and a sound effect; a movement detection section (for example, amovement detection unit 1200 shown in FIG. 8) for detecting a movementof the player; a player movement display section (for example, a gameoperation unit 1010, a player image generation unit 1034, and a datastorage medium 1500 shown in FIG. 8) for displaying the movement of theplayer detected by the movement detection section, thereon; a similaritydecision section (for example, a similarity decision unit 1016, and adata storage medium 1500 shown in FIG. 8) for comparing the assignedmovement with the movement of the player detected by the movementdetection section, to decide a similarity between the assigned dancemovement and the movement of the player; and a game operation section(for example, a game operation unit 1010, and a data storage medium 1500shown in FIG. 8) for processing a score on the basis of the similaritydecided by the similarity decision section.

In accordance with the eighth aspect of the present invention, accordingto a storage medium having a computer-executable program recordedthereon, the computer comprising a movement detection section fordetecting a movement of a player, the program comprises: a program codeof executing a predetermined game (for example, a game program 1510shown in FIG. 8); a program code of storing a predetermined assignedmovement (for example, an assignment data 1530 shown in FIG. 8); aprogram code of displaying the assigned movement (for example, a gameprogram 1510 shown in FIG. 8); a program code of inputting an input databased on a movement of the player detected by the movement detectionsection (for example, a movement input program 1540 shown in FIG. 8); aprogram code of generating at least one of a sound, a music and a soundeffect (for example, a game program 1510, and a sound data 1532 shown inFIG. 8); a program code of displaying the movement of the player on thebasis of the input data inputted (for example, a game program 1510 shownin FIG. 8); a program code of comparing the assigned movement with theinput data, to decide a similarity between the assigned movement and theinput data (for example, a game program 1510, and a decision referencedata 1536 shown in FIG. 8); and a program code of processing a score onthe basis of similarity decided (for example, a game program shown inFIG. 8).

In accordance with the ninth aspect of the present invention, a computerprogram comprises program code means for performing the steps of:storing a predetermined assigned movement for a player; displaying theassigned movement; generating at least one of a sound, a music and asound effect; detecting a movement of the player; displaying themovement of the player detected; comparing the assigned movement withthe movement of the player detected, to decide a similarity between theassigned movement and the movement of the player; and processing a scoreon the basis of the similarity decided.

According to the game apparatus, the storage medium and the computerprogram in accordance with the seventh, eighth and ninth aspects of thepresent invention, the player performs the assigned movement displayedso as to match the assigned movement, and thereby the game is advanced.

Thereafter, the movement of the player is detected and compared with theassigned movement, to decide the similarity. Based on the result ofdeciding the similarity, the score is processed, and the game progressesaccording to the score. For example, the player can enjoy comparing theplayer's movement with the assigned movement by displaying the result ofthe processing the score. Accordingly, because the player moves so as tomatch the assigned dance, the player can enjoy the experience of playingthe game using the entire body.

Further, the player can enjoy the game while listening to at least oneof the sound, the music and the sound effect generated. And further, theplayer can move to the rhythm or the tempo of the sound or the music, sothat the player can rhythmically move. Furthermore, the player movesrhythmically to the sound, the music or the sound effect, thereby otherpeople can enjoy watching the player's movement and also be motivated toplay the game themselves.

And Further, the player can play the game while checking the player'sown movement detected and displayed. Accordingly, the player can enjoywatching the player's own movement. Further, the player can checkwhether the rhythm or the tempo of the player's movement match therhythm or the tempo of the sound or the music generated.

Furthermore, the game apparatus in accordance with the seventh aspect ofthe present invention, can detect the player's movement withoutrequiring a complicated image processing operation because the movementdetection section detects the player's movement. That is, the gameapparatus is constructed so as to be different from a conventionalflag-raising type video game apparatus that detects the player'smovement based on the flags held in the player's hands, and directlydetects the movement of the player's entire body. Accordingly, theplayer can enjoy the experience of playing the game with the entirebody.

Preferably, a game apparatus in accordance with the first aspect of thepresent invention, further comprises an image display device (forexample, a display unit 1300 shown in FIG. 8, and a display 1730 shownin FIG. 1), wherein the movement detection section detects a movement ofthe player, to output an input data based on the movement detected tothe player movement display section; and the player movement displaysection displays the movement of the player on the image display device,on the basis of the input data outputted from the movement detectionsection.

According to the game system as described above, the movement of theplayer detected by the movement detection section is inputted as aninput data. That is, the movement of the player detected by the movementdetection section is outputted not as an image (a moving image) but asan input data to the player movement display section. Thereby, theplayer movement display section displays the movement of the player onthe image display section based on the input data. Accordingly, becausethe movement of the player is outputted to the player movement displaymeans as an input data, a process related to displaying the movement ofthe player can be eliminated. It should be noted that the “input data”as used here is a vector data for the movement of each part (forexample, arms and legs, a torso, a head and so on) of the player.

Preferably, according to a game apparatus in accordance with the firstaspect of the present invention, the player movement display sectiondisplays a character that moves according to the movement of the playerdetected by the movement detection section, thereon.

As described above, because the movement of the player is displayed byusing the character, the player can conform the player's own movementwith interest.

Preferably, according to a game apparatus in accordance with the fourthaspect of the present invention, the assigned movement matches at leastone of the sound, the music and the sound effect, generated by the soundgeneration section.

As described above, because the assigned movement is matched to thesound, the music or the sound effect generated, the player can recognizethe tempo or the rhythm of the assigned movement from the sound and movemore easily. Accordingly, even beginner players can enjoy playing thegame. Further, the player can simulate the assigned movement accordingto the tempo of the sound, the music or the sound effect rhythmically.

Preferably, according to a game apparatus in accordance with the fourthaspect of the present invention, the assigned movement matches a tempoof at least one of the sound, the music and the sound effect, generatedby the sound generation section.

As described above, because the assigned movement matches the tempo ofthe sound, the music or the sound effect, the player can perform theassigned movement according to the tempo of the sound, the music or thesound effect. Accordingly, even beginner players can thus enjoy playingthe game.

Preferably, a game apparatus in accordance with the first and fourthaspects of the present invention, further comprises a tempo displaysection (for example, a game operation unit 1010, and a game imagegeneration unit 1032 shown in FIG. 8) for displaying a tempo of theassigned movement, thereon.

As described above, because the tempo of the assigned movement isdisplayed on the tempo display section, the player can move according tothe tempo of the assigned movement. Accordingly, even beginner playerscan enjoy playing the game.

Preferably, a game apparatus in accordance with the first and fourthaspects of the present invention, further comprises: a game levelsetting section (for example, a game level setting unit 1022 shown inFIG. 8) for setting a game level of difficulty, wherein at least one ofa tempo of the assigned movement, a type of the assigned movement andthe similarity decided by the similarity decision section is changed onthe basis of the game level of difficulty set by the level settingsection.

According to the game apparatus as described above, because at least oneof the tempo of the assigned movement, the type of the assigned movementand the similarity decided by the similarity decision section is changedaccording to the game level of difficulty set by the game level settingsection, the wider range of players can play the game. For example, if aplayer thinks to be poor at exercise, the player can set a lower gamelevel of difficulty, and if the player feels to be good at exercise, theplayer can set a higher game level of difficulty.

Further, preferably, in the case wherein the faster the tempo of theassigned movement is, the more difficult to perform the assignedmovement is, when the game level of difficulty is raised the tempo ofthe assigned movement is made higher, and when the game level ofdifficulty is lowered the tempo of the assigned movement is made slower.On the other hand, in the case wherein the slower the tempo of theassigned movement is, the more difficult to perform the assignedmovement is, when the game level of difficulty is raised the tempo ofthe assigned movement is made slower, and when the game level ofdifficulty is lowered the tempo of the assigned movement is made higher.

And further, if the game level of difficulty is high, it is preferableto change the type of the assigned movement that is relatively difficultto perform. Conversely, if the game level of difficulty is low, it ispreferable to change the type of the assigned movement that isrelatively easy to perform.

Furthermore, if the game level of difficulty is high, it is preferableto change the result of deciding the similarity by the similaritydecision section so that the result is more demanding. Conversely, ifthe game level of difficulty is low, it is preferable to change theresult of deciding the similarity by the similarity evaluation means sothat the result is less demanding.

Preferably, according to a game apparatus in accordance with the firstand fourth aspects of the present invention, at least one of a tempo ofthe assigned movement, a type of the assigned movement and thesimilarity decided by the similarity decision section is changed on thebasis of the similarity decided by the similarity decision section.

As described above, because the tempo of the assigned movement, the typeof the assigned movement and the similarity decided by the similaritydecision section can be changed based on the similarity, evenexperienced players that have become accustomed to the game can continueenjoy playing the game. That is, because the tempo of the assignedmovement is not simple and the type of the movement is monotonous,players can continue to play the game seriously.

Furthermore, because the similarity decided is changed, the scoreprocessed by the game operation section differs even for differentplayers performing the same movement, so that the game will not becomemonotonous. Accordingly, players can continue to play the gameseriously.

Preferably, according to a game apparatus in accordance with the firstand fourth aspects of the present invention, the similarity decisionsection compares the assigned movement with the movement of the playerdetected by the movement detection section, at a predetermined time ofthe assigned movement, to decide the similarity between the assignedmovement and the movement of the player; and the game apparatus furthercomprises a timing notice section (for example, a game operation unit1010, an assigned dance setting unit 1014, a game image generation unit1032, and a sound generation unit 1040 shown in FIG. 8) for indicatingthe predetermined time to the player by a visual effect and by a visualeffect or an auditory effect, respectively.

As described above, because the predetermined time to decide thesimilarity is indicated by the timing notice section, the player canconform the time wherein the player's movement is decided. Therefore,the player can be stimulated to perform the particular movement at thepredetermined time, so that the player can move rhythmically.Accordingly, the player can play the game with interesting.

Preferably, according to a game apparatus in accordance with the firstand fourth aspects of the present invention, the similarity decisionsection compares the assigned movement with the movement of the playerdetected by the movement detection section, at a predetermined time ofthe assigned movement, to decide the similarity between the assignedmovement and the movement of the player; and the game apparatus furthercomprises a decision display section (for example, a game operation unit101, an assigned dance setting unit 1014, and a game image generationunit 1032 shown in FIG. 8) for displaying a decision according to thesimilarity decided at the predetermined time by the similarity decisionsection.

As described above, because the decision is displayed according to thesimilarity at the predetermined time, the player can conform the resultof comparing the player's own movement with the assigned movement.Accordingly, the player can play the game so as to improve the decision,so that the player can enjoy the game.

Preferably, according to a game apparatus as described above, thedecision display section is a gauge display section (for example, a gameoperation unit 1010, and a game image generation unit 1032 shown in FIG.8) for displaying a gauge that fluctuates according to the decision ofthe similarity decided at the predetermined time by the similaritydecision section.

As described above, the player can play the game while watching thegauge fluctuating according to the decision of the similarity at thepredetermined time. That is, the player can play the game whilecomparing the player's own movement with the assigned movement.Accordingly, players can concentrate on the game while playing game withthe desire to improve the similarity with the assigned movement.

Preferably, according to a game apparatus as described above inaccordance with the fourth aspect of the present invention, thepredetermined time is changed according to a tempo of at least one ofthe sound, the music and the sound effect, generated by the soundgeneration section.

As described above, because the predetermined time can be changedaccording to the tempo of the sound, the music or the sound effect, therhythm of the assigned movement and the predetermined time wherein thesimilarity is decide can be matched to the similarity.

Preferably, according to a game apparatus in accordance with the firstand fourth aspects of the present invention, the game operation sectioncalculates the score on the basis of the similarity decided by thesimilarity decision section.

As described above, because the score is calculated on the basis of thesimilarity decided by the similarity decision section, the player canobjectively decide the comparison between the player's movement and theassigned movement by the score.

Preferably, according to a game apparatus in accordance with the firstand fourth aspects of the present invention, the game operation sectioncalculates the score on the basis of the similarity decided by thesimilarity decision section, to calculate a cumulative total of thescore calculated; and the game apparatus further comprises a scoredisplay section (for example, a game image generation unit 1010, and agame image generation unit 1032 shown in FIG. 8) for displaying thecumulative total of the score, thereon.

As described above, because the cumulative total of the score isdisplayed, the player can decide the similarity between the player'smovement and the assigned movement. That is, the player can confirm theplayer's own score during the game. Accordingly, the player can play thegame with the desire to become even better.

Preferably, according to a game apparatus in accordance with the firstand fourth aspects of the present invention, the assigned movementdisplay section displays the assigned movement before the similaritydecision section decides the similarity.

As described above, because the assigned movement is displayed beforethe similarity decision section decides the similarity, that is, beforethe game operation section performs the score, the player can rememberthe assigned movement before beginning to perform the moment. In otherwords, after the player has first the opportunity to remember theassigned movement, the player copies the assigned movement and thesimilarity between the player's movement and the assigned movement, isthen decided. Accordingly, the player can enjoy the game without thegame becoming simply a game of reflexes.

Preferably, a game apparatus in accordance with the first and fourthaspects of the present invention, further comprises a movement startnotice section (for example, a game operation unit 1010, a game imagegeneration unit 1032, and a sound generation unit 1040 shown in FIG. 8)for indicating when the player starts moving to the player by a visualeffect and by a visual effect or an auditory effect, respectively.

As described above, because the player is indicated when to startmoving, the player can relatively easily begin playing game play.

It should be noted that the time for starting to play the game can beone, for example, when the similarity decision section starts decidingthe similarity.

Preferably, accordance to a game apparatus in accordance with the firstand fourth aspects of the present invention, the assigned movement is adance.

As described above, because the assigned movement is a dance, the playercan more easily perform rhythmically. Accordingly, players can enjoyplaying the game.

Preferably, according to a game apparatus in accordance with the firstand fourth aspects of the present invention, the movement detectionsection comprises an artificial retina chip.

As described above, the artificial retina chip can perform such as animage processing as an image detection, an edge detection, a patternmatching, a resolution varying process, a gaze and so on, by own chipwithout other image processing device. Accordingly, the game apparatuscan simplified and require a lower manufacturing cost.

Preferably, according to a game apparatus in accordance with the firstand fourth aspects of the present invention, the movement detectionsection detects the movement of the player from a light received througha visible spectrum cut-off filter (for example, a visible spectrumcut-off filter 1740 shown in FIG. 1).

As described above, because the player's movement is detected byreceiving the light through the visible spectrum cut-off filter, themovement detection section receives a light from which a visiblespectrum light has been removed. More specifically, there are situationsin which the player's movement cannot be detected because ofinterference from random outside light and so on. However, it ispossible to detect only the player's movement without a process forremoving interference from random outside light and so on, by receivingthe light passed through a visible spectrum cut-off filter.

Accordingly, the movement of the player can be detected withoutinterfering the progress of the game, and thereby it is possible todetect the similarity on the basis of the detected movement.

It should be noted that the visible spectrum cut-off filter ispreferably disposed directly before a photo-detection unit between thephoto-detection unit and the player. Consequently, it is possible toreliably detect only the player's movement without being affected byinterference from random outside light and so on.

Preferably, a game apparatus in accordance with the first and fourthaspects of the present invention, further comprises a wall member (forexample, a back panel 1780 shown in FIG. 1) disposed upright in a lineof the movement detection section and the player.

As described above, because the wall member is disposed upright in aline of the movement detection section and the player, the movementdetection section does not detect the movements of objects or peoplebehind the wall member. That is, the movement detection section canreliably detect only the player's movement. Accordingly, interferencewith the game play is prevented, and the player can relax andconcentrate on the game.

Preferably, a game apparatus as described above, further comprises aprojection suppression section for suppressing a projection of a shadowof the player on the wall member.

As described above, while an ambient artificial lighting and a naturallight can project a shadow of the player onto the wall member, theprojection suppression section prevents the player's shadow fromprojecting on the wall member, so that the movement detection sectioncan reliably detect only the player's movements without detecting theshadow's movement. Accordingly, the player can relax and concentrate onthe game without interfering with the game play.

Herein, according to the projection suppression section, it is possibleto suppress the projection of the player's shadow to the wall member,for example, by coloring a surface of the wall member such a dark coloras a gray, a black, a dark blue and so on. Further, it is possible tosuppress the projection of the player's shadow by illuminating theplayer with a direct light from the side of the player, that is, fromthe direction orthogonal to the line joining the player and movementdetection section, so as not to illuminate with a light to the wallmember.

Further, this type of projection suppression section can prevent theshadows of not only the player but also other people near the gameapparatus from projecting onto the wall member.

In accordance with the tenth aspect of the present invention, a gameapparatus (for example, a game apparatus 2700 shown in FIG. 1) forexecuting a predetermined game, comprises: a storage section (forexample, an assignment data 2530 shown in FIG. 24) for storing apredetermined assigned movement; a movement detection section (forexample, an image sensor 2200 shown in FIG. 24) for detecting a movementof a player; and a similarity decision section (for example, a movementspecifying unit 2017, and a similarity decision unit 2016 shown in FIG.24) for deciding a similarity between the movement of the playerdetected by the movement detection section and the predeterminedassigned movement stored in the storage section, on the basis of atleast one of a direction, a magnitude and a speed of the movement of theplayer.

In accordance with the eleventh aspect of the present invention,according to a storage medium having a computer-executable programrecorded thereon, the computer comprising a movement detection section(for example, an image sensor 2200 shown in FIG. 24) for detecting amovement of a player, the program comprises: a program code of executinga predetermined game (for example, a game program 2510 shown in FIG.24); a program code of storing a predetermined assigned movement (forexample, a decision reference data 2536 shown in FIG. 24); and a programcode of deciding a similarity between the movement of the playerdetected by the movement detection section and the predeterminedassigned movement, on the basis of at least one of a direction, amagnitude and a speed of the movement of the player (for example, a gameprogram 2510 shown in FIG. 24, and steps S204 to S214 shown in FIG.34A).

In accordance with the twelfth aspect of the present invention, acomputer program comprises program code means for performing the stepsof: executing a predetermined game; storing a predetermined assignedmovement; detecting a movement of a player; and deciding a similaritybetween the movement of the player detected and the predeterminedassigned movement, on the basis of at least one of a direction, amagnitude and a speed of the movement of the player.

According to the game apparatus, the storage medium and the computerprogram in accordance with the tenth, eleventh and twelfth aspects ofthe present invention, the movement of the player is detected, and themovement of the player detected and the predetermined assigned movementis compared and decided.

Consequently, because the player is freed from the necessity fordepressing the switch during the game play in order to discriminate thatthe player has moved, the player can enjoy the physical game using theentire body without being restricted by the inputting action. Further,because the similarity is decided with the elements of the direction,the magnitude and the speed of the movement of the player, thesubjective decision on whether or not the movement of the player is goodcan be objectively made. Herein, it is possible to enhance the effect ofthe present invention that the predetermined game is exemplified by thedance game.

Preferably, according to a game apparatus in accordance with the tenthaspect of the present invention, the similarity decision section decidesthe similarity between the movement of the player and the predeterminedassigned movement corresponding to a predetermined time thereof, everythe predetermined time of the assigned movement.

As described above, the similarity of the movement of the player isdecided every the predetermined time. Therefore, it is unnecessary todecide the movement of the player at all times so that the processingcan be lightened. Further, by accumulating the similarities every thepredetermined time, i.e., the similarities decided by using thedirection, the magnitude and the speed as the elements, the entiremovement of the player can be objectively decided.

Preferably, a game apparatus in accordance with the tenth aspect of thepresent invention, further comprises: a timing notice section forindicating a predetermined time by a visual effect or an auditoryeffect; and a timing decision section for deciding whether the movementof the player detected by the movement detection section coincides withthe predetermined time, or not.

As described above, for example, it is possible to reduce the score, ifthe movement is identical to the predetermined assigned movement but isnot timed. Here, because it is severe against the player to strictlydecide whether the movement is timed, it is arbitrary to provide a widthof about 0.2 seconds for the allowable time. Further, the coincidence onhow to be timed is calculated so that, for example, the value “1” iscalculated for the just timed case, and the value 110.811 is calculatedfor a discrepancy of 0.05 seconds. This coincidence may be added as acoefficient to the decision result of the similarity. In this case, itis possible to relieve a wrong movement in which the right arm iserroneously raised when the left arm should be raised.

Preferably, a game apparatus as described above, further comprises: amovement specifying section (for example, a movement specifying unit2017 shown in FIG. 24) for specifying the movement of the player to bedecided on the similarity, wherein the movement of the player isspecified from a plurality of movements of the player concurrentlydetected by the movement detection section when being decided tocoincide with the predetermined time by the timing decision section;wherein the similarity decision section decides the similarity betweenthe movement of the player specified by the movement specifying sectionand the predetermined assigned movement.

As described above, in the case wherein there has been a plurality ofmovements of the player actions, the movement to be decided on thesimilarity can be specified with reference to the decision reference onwhether or not the movement is coincided with the predetermined time.Accordingly, in other words, the movement not coincided with thepredetermined time can be eliminated from the object to be decided onthe similarity.

Preferably, a game apparatus in accordance with the tenth aspect of thepresent invention, further comprises: a movement specifying section (forexample, a movement specifying unit 2017 shown in FIG. 24) forspecifying the movement of the player to be decided on the similarity,wherein the movement of the player is specified from a plurality ofmovements of the player concurrently detected by the movement detectionsection, on the basis of at least one of the direction, the magnitudeand the speed of each of the movements of the player; wherein thesimilarity decision section decides the similarity between the movementof the player specified by the movement specifying section and thepredetermined assigned movement.

As described above, even in the case wherein there are a plurality ofmovements of the player, the movement to be decided on the similaritycan be specified to decide the similarity. Herein, the element to beused for specifying the movement may be changed according to theassigned movement such that it is referred to the magnitude when a largemovement is demanded, and such that it is referred to the speed when aquick movement is demanded. On the other hand, by specifying themovement in a direction different from that of the assigned movement, itis arbitrary to lower the decision for the movement of the player.

Preferably, according to a game apparatus as described above, themovement specifying section specifies the movement of the player to bedecided on the similarity, for each of a plurality of detection regionsdivided from a detection range of the movement detection section, andthe similarity decision section decides the similarity between themovement of the player specified by the movement specifying section andthe predetermined assigned movement corresponding to the detectionregion, every the detection region.

Preferably, according to a game apparatus as described above, thesimilarity decision section decides the similarity, by considering themovement of the player extended over a plurality of detection regions,every the detection regions extended.

As described above, for example, in the case wherein the decisionreference of the similarity is the magnitude of the movement, even ifthe movement of the player is extended over the detection regions, themovement can be decided considering the magnitudes of the individualmovement in the detection regions. Accordingly, it is possible toprevent the large movement from being erroneously decided.

Preferably, according to a game apparatus in accordance with the tenthaspect of the present invention, the similarity decision section decidesthe similarity between the movement of the player and the predeterminedassigned movement corresponding to each of a plurality of detectionregions (for example, the 1^(st) to 4^(th) regions shown in FIG. 28)divided from a detection range of the movement detection section, everythe detection region.

Preferably, according to a storage medium having a computer-executableprogram recorded thereon, in accordance with the eleventh aspect of thepresent invention, the program further comprises: a program code ofdeciding a similarity between the movement of the player and thepredetermined assigned movement corresponding to each of a plurality ofdetection regions divided from a detection range of the movementdetection section, every the detection region.

Preferably, a computer program in accordance with the twelfth aspect ofthe present invention, further comprises program code means forperforming the step of: deciding a similarity between the movement ofthe player and the predetermined assigned movement corresponding to eachof a plurality of detection regions divided from a detection rangedetected, every the detection region.

According to the game apparatus, the storage medium the computerprogram, as described above, the similarity between the movement of theplayer and the assigned movement can be decided on how the similarity ofthe entire detection regions is, every the detection region.Consequently, it is unnecessary that each movement of the player isdecided, so that the process of deciding the similarity can belightened. Further, the efficiency of the processing can be improved byspecifying the detection regions to be decided.

Preferably, a game apparatus as described above, further comprises aby-region decision display section for displaying a decision accordingto the similarity decided every the detection region by the similaritydecision section.

As described above, the player can recognize which of detection regionsthe movement has been good or bad in, to know a more specific decisionresult.

Preferably, according to a game apparatus as described above, thedetection region is changed according to a predetermined conditiondetected by the movement detection section.

Preferably, according to a game apparatus in accordance with the tenthaspect of the present invention, the similarity decision section changesthe similarity decided according to a predetermined condition detectedby the movement detection section.

Preferably, according to a game apparatus as described above, thepredetermined condition relates to a body shape of the player detectedby the movement detection section.

As described above, the similarity decided and the detection region canbe automatically changed without any operation input of the player,according to such a predetermined condition detected by the movementdetection section, as the body shape (for example, the height or thelength of the limbs) of the player or the position of the player.Accordingly, it is possible to correct the differences and so on in thedecision result of the similarity due to the body shape of the player.Herein, the change in the detection region contains the change in thenumber of the detection regions, in the magnitude of the detectionregions and in the positions of the detection regions.

Preferably, according to a game apparatus in accordance with the tenthaspect of the present invention, the movement detection sectioncomprises an artificial retina chip for detecting the movement of theplayer.

As described above, the movement detection section detects the movementof the player with the artificial retina chip. As a result, theprocessing such as the characteristic extraction and so on of thedetection can be facilitated and distributed into the entire apparatus,so that the processing can be speeded up as the entire apparatus.

In accordance with the thirteenth aspect of the present invention, agame apparatus (for example, a game apparatus 3700 shown in FIG. 1)comprises; a movement detection section (for example, an image sensor3200 shown in FIG. 42) for detecting a movement of a player; a gameexecution section (for example, a similarity decision unit 3016 shown inFIG. 42) for executing a predetermined game by comparing the movement ofthe player detected by the movement detection section with apredetermined assigned movement; and a hint outputting section (forexample, a character movement setting unit 3012, and a viewpoint controlunit 3019 shown in FIG. 42) for outputting a hint corresponding to thepredetermined assigned movement by a visual effect or an auditoryeffect.

In accordance with the fourteenth aspect of the present invention,according to a storage medium (for example, a data storage medium 3500shown in FIG. 42) having a computer-executable program recorded thereon,the program comprises: a program code of detecting a movement of aplayer; a program code of executing a predetermined game by comparingthe movement of the player detected with a predetermined assignedmovement (for example, a game program 3510 shown in FIG. 42); and aprogram code of outputting a hint corresponding to the predeterminedassigned movement by a visual effect or an auditory effect (for example,a character movement data 3534, a control pattern data 3538, and avirtual camera control data 3540 shown in FIG. 42).

In accordance with the fifteenth aspect of the present invention, acomputer program comprising program code means for performing the stepsof: detecting a movement of a player; executing a predetermined game bycomparing the movement of the player detected with a predeterminedassigned movement; and outputting a hint corresponding to thepredetermined assigned movement by a visual effect or an auditoryeffect.

According to the game apparatus, the storage medium and the computerprogram in accordance with the thirteenth, fourteenth and fifteenthaspects of the present invention, when the player performances amovement based on the hint corresponding to the predetermined assignedmovement by a visual effect or an auditory effect, the movement of theplayer is detected and reflected to the game. Accordingly, thepredetermined assigned movement and the movement of the player are notlimited to a constant movement, so that it is possible to realize a bodysensing game requiring the player to use the entire body thereof.Furthermore, the game level of difficulty or the content of the game canbe readily altered by varying the timing of outputting the hint, or bychanging the contents of the hint.

Preferably, according to a game apparatus in accordance with thethirteenth aspect of the present invention, the hint outputting sectionoutputs the hint by displaying a character performing a movement.

Preferably, according to a storage medium having a computer-executableprogram recorded thereon, in accordance with the fourteenth of thepresent invention, the program further comprises: a program code ofdisplaying a character performing a movement to be outputted as thehint.

Preferably, a computer program in accordance with the fifteenth of thepresent invention, further comprises program code means for performingthe step of: displaying a character performing a movement to beoutputted as the hint.

According to the game apparatus, the storage medium and the computerprogram as described above, because the hint is provided through thecharacter performing the movement, it is possible to provide an effectin that the player can more easily recognize the movements to beperformed as compared with a case in which the player has to followvoice instructions or instructions displayed on the screen.

Preferably, according to a game apparatus as described above, the hintoutputting section does not display the character before amovement-start instruction is given to the player, when a predeterminedgame level of difficulty is selected by the player.

As described above, because the player is not provided with the hintconcerning the assigned movement before the player starts performing themovement, the player has to move reflectively by viewing the movement ofthe character displayed after the movement-start instruction.Accordingly, the game level of difficulty increases, so that it ispossible that the skilled players can play a more interesting game.

Preferably, according to a game apparatus as described above, the hintoutputting section displays the character facing to the player andperforming the movement laterally inversed the predetermined assignedmovement.

Preferably, according to a storage medium having a computer-executableprogram recorded thereon, as described above, the program furthercomprises: a program code of displaying the character facing to theplayer and performing the movement laterally inversed the predeterminedassigned movement.

Preferably, a computer program as described above, further comprisesprogram code means for performing the step of: displaying the characterfacing to the player and performing the movement laterally inversed thepredetermined assigned movement.

According to the game apparatus, the storage medium and the computeprogram as described above, because the character performs the movementlaterally inversed the predetermined assigned movement, the player mayintuitively determine the assigned movement from the left and rightsides of the movement as viewed, and can reflectively provide aresponse. Accordingly, it is possible to reduce the time period from thetime wherein the hint is displayed by the character performing themovement to the time wherein the player performs actually the movement.

Preferably, according to a game apparatus as described above, the hintoutputting section displays the character before a movement-startinstruction is given to the player.

Preferably, according to a storage medium having a computer-executableprogram recorded thereon, as described above, the program furthercomprises: a program code of displaying the character before amovement-start instruction is given to the player.

Preferably, a computer program as described above, further comprisesprogram code means for performing the step of: displaying the characterbefore a movement-start instruction is given to the player.

According to the game apparatus, the storage medium and the computerprogram as described above, because the hint concerning the assignedmovement is outputted by displaying the character prior to themovement-start instruction to the player, the player can obtain theinformation on the assigned movement before the player starts performingthe movements. Accordingly, beginners of the game can play the game withmore interesting.

Preferably, according to a game apparatus as described above, the hintoutputting section displays the character after a movement-startinstruction is given to the player.

Preferably, according to a storage medium having a computer-executableprogram recorded thereon, as described above, the program furthercomprises: a program code of displaying the character after amovement-start instruction is given to the player.

Preferably, a computer program as described above, further comprisesprogram code means for performing the step of: displaying the characterafter a movement-start instruction is given to the player.

According to the game apparatus, the storage medium and the computerprogram as described above, because the hint concerning the assignedmovement is outputted by displaying the character while the playerperforming the movement, the player can simulate the assigned movementbased on the movement of the character. Accordingly, the game level ofdifficulty may be lowered, to enhance the amusement of the game forbeginners.

Preferably, according to a game apparatus as described above, the hintoutputting section changes the movement of the character to bedisplayed, according to a game level of difficulty selected by theplayer or the movement of the player detected by the movement detectionsection.

As describe above, because the game level of difficulty wherein theplayer is presently playing can be changed, by changing the movement ofthe character, the game level of difficulty can be changed easily.

Preferably, according to a game apparatus as described above, the hintoutputting section displays the character performing the movement havingno association with the predetermined assigned movement, when apredetermined game level of difficulty is selected by the player.

As described above, because the character shows a movement having noassociation with the predetermined assigned movement while the player isperforming the movement, the player may be mislead by the movement ofthe character. Therefore, it would be more difficult for the player toperform the assigned movement correctly. Accordingly, the game level ofdifficulty increases, and thereby it is possible to provide a moreamusing game to skilled players.

Preferably, according to a game apparatus in accordance with thethirteenth aspect of the present invention, the hint outputting sectiondoes not output the hint corresponding to the predetermined assignedmovement by a visual effect or an auditory effect, when a predeterminedinstruction is inputted by the player.

As described above, because the player is not given the hint concerningthe assigned movement when the predetermined instruction is inputted,the player can freely move to reflect the movements to the game.Accordingly, the player feels as if the player's movement is good orpoor is decided, so that the player can get a kind of amusement that isdifferent from the game in which the player has to copy the assignedmovement. Consequently, the contents of the game can be changed invarious ways.

Preferably, according to a game apparatus in accordance with thethirteenth aspect of the present invention, the movement detectionsection comprises an artificial retina chip for detecting the movementof the player.

As described above, because the player's movement is detected by usingthe artificial retina chip, the process such as the feature extractionprocess and so on, for detecting the player's movement can be performedon the side of the artificial retina chip. Accordingly, the totalprocessing load of the apparatus can be more distributed, the processingspeed of the apparatus as a whole can be further increased.

Preferably, according to a game apparatus in accordance with thethirteenth aspect of the present invention, the predetermined assignedmovement is a movement which synchronizes with a predetermined tempo ofat least one of a music, a sound, a sound effect and a video.

As described above, because the player moves in synchronization with thetempo of the music, the sound, the sound effect or the video, thepleasure in performing the movements can be increased. Further, becausethe player has to move with the tempo, the game level of difficulty canreadily be changed by changing the tempo of the music, the sound, thesound effect or the like.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will become more fully understand from thedetailed description given hereinafter and the accompanying drawingwhich are given by way of illustration only, and thus are not intendedas a definition of the limits of the present invention, and wherein:

FIG. 1 is an external front view of a game apparatus 1700 (2700, 3700)as an arcade game machine to which the present invention is applied;

FIG. 2 is a sectional view of the game apparatus 1700 (2700, 3700) as anarcade game machine to which the present invention is applied;

FIG. 3 is a flow chart showing the operation of the game apparatus 1700according to the first embodiment of the present invention;

FIG. 4 is a view showing an exemplary sound selection screen presentedin the game apparatus 1700;

FIG. 5 is a view showing an exemplary character selection screenpresented in the game apparatus 1700;

FIG. 6 is a view showing an exemplary game level selection screenpresented in the game apparatus 1700;

FIG. 7 is a view showing an exemplary screen presented in the gameapparatus 1700, while the game is being played;

FIG. 8 is a functional block diagram of the game apparatus 1700;

FIG. 9 is a view showing an example of a result of detecting a player'smovement;

FIG. 10 is a diagram showing an example of a player's movement data;

FIG. 11 is a diagram showing an example of a decision reference data1536;

FIG. 12 is a diagram showing an example of a character movement data1534;

FIG. 13 is a flow chart of a process of the game apparatus 1700;

FIG. 14 is a view showing an exemplary screen in a memorizing mode,presented in the game apparatus 1700;

FIG. 15 is a view showing an exemplary screen in a dance mode, presentedin the game apparatus 1700;

FIG. 16 is a view showing an exemplary screen in the memorizing mode,presented in the game apparatus 1700;

FIG. 17 is a view showing an exemplary screen in the dance mode,presented in the game apparatus 1700;

FIG. 18 is a block diagram showing an exemplary hardware configurationcapable of realizing the game apparatus 1700;

FIG. 19A and FIG. 19B are views showing some possible variations of agame apparatus to which the present invention is applied;

FIG. 20 is a view showing an exemplary screen presented in the gameapparatus 1700;

FIG. 21 is a view showing an exemplary screen in a memorizing mode,presented in the game apparatus 2700 according to the second embodimentof the present invention;

FIG. 22 is a view showing an exemplary screen in a dance mode, presentedin the game apparatus 2700;

FIG. 23 is a view showing an exemplary screen in a dance mode, presentedin the game apparatus 2700;

FIG. 24 is a functional block diagram of the game apparatus 2700;

FIG. 25 is a diagram showing an example of a character movement data2534;

FIG. 26 is a diagram view showing an example of a decision reference2536;

FIG. 27 is a view showing a range which can be detected by an imagesensor 2200;

FIG. 28 is a view showing a first region to a fourth region;

FIG. 29 is a diagram showing an example of a player's movement data;

FIG. 30A is a view showing one example of the case in which a playerperforms a plurality of movements, and FIG. 30B is a view showing thevectors of the player's movements shown in FIG. 30A, detected by theimage sensor 2200;

FIG. 31A is a view showing one example of the case in which a playerperforms a plurality of movements in the same region, and FIG. 31B is aview showing the vectors of the player's movements shown in FIG. 31A,detected by the image sensor 2200;

FIG. 32 is a view for explaining a decision of similarity on timedmovements;

FIG. 33 is a view for explaining a decision of similarity on movementswhich are not timed but within an allowable time;

FIG. 34A and FIG. 34B are flow charts of a process of the game apparatus2700;

FIG. 35 is a block diagram showing an exemplary hardware configurationcapable of realizing the game apparatus 2700;

FIG. 36 is a view showing an exemplary screen of the case in which thedecision result of each region is displayed;

FIG. 37 is a view showing an exemplary screen in a memorizing mode,presented in the game apparatus 3700 according to the third embodimentof the present invention;

FIG. 38 is a view showing an exemplary game level selection screenpresented in the game apparatus 3700;

FIG. 39 is a view showing an exemplary screen presented in the gameapparatus 3700, before a dance;

FIG. 40 is a view showing an exemplary screen presented in the gameapparatus 3700, during a dance;

FIG. 41A and FIG. 41B are views showing a relationship between amovement of an instructor character 3956 and a movement of the player1800;

FIG. 42 is a functional block diagram of the game apparatus 3700;

FIG. 43 is a diagram showing an example of a control pattern data 3538;

FIG. 44 is a diagram showing an example of a character movement data3534;

FIG. 45A and FIG. 45B are views for illustrating a virtual camera 3542in an object space;

FIG. 46 is a diagram showing an example of a virtual camera control data3540;

FIG. 47 is a view showing an exemplary display of the instructorcharacter 3956 presented in the game apparatus 3700;

FIG. 48 is a view showing an exemplary display of the instructorcharacter 3956 presented in the game apparatus 3700;

FIG. 49 is a view showing an exemplary display of the instructorcharacter 3956 presented in the game apparatus 3700;

FIG. 50 is a diagram showing an example of a decision reference data3536;

FIG. 51 is a flow chart of a process of the game apparatus 3700;

FIG. 52 is a block diagram showing an exemplary hardware configurationcapable of realizing the game apparatus 3700; and

FIG. 53 is a view showing an exemplary screen presented in the gameapparatus 3700, during a dance.

PREFERRED EMBODIMENT OF THE INVENTION

Hereinafter, a preferred embodiment of the present invention will beexplained in detail with reference to figures.

The present invention is described below as applied to a dance game byway of example only, and it will be obvious to one with ordinary skillin the related art that the present invention shall not be limited tosuch applications.

First, a game apparatus 1700 suitable to games that can be experiencedby a player using an entire body to play a game, will be explainedaccording to the first embodiment of the game apparatus of the presentinvention, with reference to FIGS. 1 to 20.

Next, a game apparatus 2700 which can progress a game and process ascore according to a movement using an entire body without restrictingan inputting action, will be explained according to the secondembodiment of the game apparatus of the present invention, withreference to FIGS. 1, 2 and 21 to 36.

And next, a game apparatus 3700 which can realize a body sensing gamerequiring a player to use an entire body and vary contents of the gameby outputting hints, will be explained according to the third embodimentof the game apparatus of the present invention, with reference to FIGS.1, 2 and 37 to 53.

First Embodiment

The game apparatus 1700 suitable to games that can be experienced by aplayer using his entire body to play a game, will be explained accordingto the first embodiment of the game apparatus of the present invention,with reference to FIGS. 1 to 20.

FIGS. 1 and 2 are exemplary external views showing the game apparatus1700 as an arcade game machine according to the first embodiment of thegame apparatus of the present invention. FIG. 1 is a front view showingan external of the game apparatus 1700, and FIG. 2 is a verticalsectional view from the side of the game apparatus 1700.

Further, in FIGS. 1 and 2, the reference numerals of each part of thegame apparatus 2700 and the game apparatus 3700 according to the secondand third embodiments are illustrated in parentheses, corresponding toeach part of the game apparatus 1700 according to the first embodiment.

As shown in FIG. 1, this game apparatus 1700 comprises substantiallyhorizontal a dance stage 1770, a housing 1750 standing upright at thefront of the dance stage 1770, a back panel 1780 standing upright at theback of the dance stage 1770, an artificial retina camera 1720, adisplay (image display device) 1730, operating buttons 1710, a speaker1760 and a light 1790 and so on, wherein the artificial retina camera1720, the display 1730, the operating buttons 1710, the speaker 1760 andthe light 1790 are provided at the housing 1750. Note that thisartificial retina camera 1720 is the movement detection means fordetecting the movement of the player 1800.

The player 1800 dances (moves) on the dance stage 1770 and enjoysplaying the dance game. The position in which the player 1800 dances, isalso indicated on the dance stage 1770. This designated dancing positionis the area in which player's movement can be detected by the artificialretina camera 1720.

The operating buttons 1710 can be alternatively disposed near where theplayer 1800 dances. Buttons operated with the feet could also beprovided on the dance stage 1770 such that the player 1800 steps onthese buttons at some particular timing while dancing. Touch-screen typebuttons, for example, could also be provided near the dancing positionsuch that the player 1800 touches these buttons at some particulartiming while dancing.

The artificial retina camera 1720 and display 1730 are positioned infront of the player 1800 with the artificial retina camera 1720 abovethe display 1730. It will be obvious to one with ordinary skill in therelated art that the artificial retina camera 1720 could be placed belowthe display or on the sides of the display. Further alternatively, ahalf mirror and the artificial retina camera could be placed in front ofthe player 1800 such that movement of the player 1800 is detectedthrough the half mirror. In other words, the display and artificialretina camera can be arranged as desired insofar as the player 1800 cansee the display and the artificial retina camera can capture images ofthe movement of the player 1800 while the player 1800 is playing thegame.

The speaker 1760 can also be placed as desired, including, for example,above the dance stage 1700 or in the back panel 1780.

The artificial retina camera 1720 has an artificial retina chip fordetecting the player 1800. This artificial retina chip performs an imagedetection process for detecting an image, and a feature extractionprocess for detecting particular features of the detected image. Thisfeature extraction process could be, for example, an edge detectionprocess for detecting the contours of a subject in the image; aresolution varying process for reading pixel groups; a pattern matchingprocess for extracting a subject from the image; or a random accessprocess for detection only in a particular part of the image. Based onthe image processed by the artificial retina chip, the artificial retinacamera 1720 then detects player 1800 movement (further described below).

Because the artificial retina chip can accomplish the image detection,the edge detection, the pattern matching, the resolution varyingprocess, the gaze, or other feature extraction process without otherimage processing, it simplifies the configuration of a game apparatus1700 according to this preferred embodiment of the present invention andmakes it possible to reduce the manufacturing cost of the game apparatus1700.

A visible light cut-off filter 1740 is also disposed between the player1800 and the photo-detection unit of the artificial retina camera 1720so that the photo-detection element of the artificial retina chipdetects only light from which light in the visible spectrum has beenfiltered out. This can be achieved by placing the visible light cut-offfilter 1740 in the housing 1750 between the player 1800 and artificialretina camera 1720.

The artificial retina camera 1720 thus detects player's movement fromlight received through the visible light cut-off filter 1740. Because alight having a wavelength of the visible spectrum is removed by thevisible light cut-off filter 1740, the artificial retina camera 1720 candetect only player movement without distortion introduced by randomoutside light, and is thus prevented from player's movement detectionerrors.

The visible light cut-off filter 1740 can also be achieved byappropriately coating the photo-detection unit of the artificial retinacamera 1720.

The back panel 1780 stands upright behind the player 1800 in lineconnecting between the artificial retina camera 1720 and player 1800. Asurface portion 1782 of the back panel 1780, which is at the side of thehousing 1750 prevents movement of people and objects passing or movingbehind the back panel 1780 from being detected by the artificial retinacamera 1720. The back panel 1780 is a primarily gray, a dark blue, orother deep shade to suppress the projection of the player's shadow ontothe back panel 1780 by an ambient natural or an artificial light. Theartificial retina camera 1720 thus will not detect movement of theplayer's shadow.

It will be obvious to one with ordinary skill in the related art thatthe back panel 1780 does not need to be a panel, but could be a screenor curtain, for example.

The game apparatus 1700 according to the first embodiment is thus a gamemachine that determines the progress and tallies the score of the dancesgame by comparing and evaluating the movement of the player 1800detected by the artificial retina camera 1720 with an assigned dance.

FIG. 3 is a flow chart of a game flow in the game apparatus 1700.

In the game apparatus 1700 according to this preferred embodiment of thepresent invention, the game starts when a coin is inserted into the gameapparatus 1700 or the start button (not shown in figures) is pressed(step S101). How to play the game is then explained using the display1730 or directions are presented through the speaker 1760 (step S102).

A sound selection screen is then presented on the display 1730, as shownin FIG. 4 (step S103). FIG. 4 shows a typical screen for selecting thesound and the music of the dance and other sound effects. The player1800 presses the next button 1950 or the back button 1952 on the soundselection screen to hear the selection of the sound, the music and theeffects that will be used when playing the game (dancing). Afterlocating the desired sound, music or effect, the player 1800 presses theplay button 1954 to start playing the desired selection. It should benoted that operating the next button 1950, the back button 1952, and theplay button 1954 can be realized by operating the operating buttons 1710or by detecting the player's movement of by means of the artificialretina camera 1720.

Next, a character selection screen is presented on the display 1730, asshown in FIG. 5 (step S104). FIG. 5 shows a typical screen for selectingthe character, displayed on the display 1730. This screen enables theplayer 1800 to select both a player character and an instructorcharacter.

Selection is aided by a prompt such as “please select your character”and so on presented on display 1730 and/or by the speaker 1760. Theplayer 1800 then uses the next button 1950 and the back button 1952 toview the different characters on the display 1730. When the desiredcharacter is found, the player 1800 presses the play button 1954 to setthe player character.

After the player character selection is made, a prompt such as “pleaseselect your instructor” and so on is presented for the player 1800 onthe display 1730 or by the speaker 1760 as noted above. The player 1800then uses the next button 1950 and the back button 1952 to view thedifferent instructor characters on the display 1730. When the desiredinstructor character is found, the player 1800 presses the play button1954 to set the instructor character.

A level selection screen for selecting the level of difficulty forplaying the game is then presented on display 1730, as shown in FIG. 6(step S105). FIG. 6 shows a typical screen for selecting the game level.The player 1800 uses the next button 1950 and the back button 1952 toselect the desired game level (difficulty level), and then presses theplay button 1954 to set the game level.

The player 1800 then starts playing the game according to the dance game(step S106). FIG. 7 shows a typical game play screen while the game isbeing played. In this dance game the player 1800 dances on the dancestage 1770 to the rhythm and the tempo of the music, the sound and thesound effects flowing from the speaker 1760, using the entire body toenjoy the dance game.

Further, the player 1800 also dances on the dance stage 1770 accordingto the movements of the instructor character 1956 shown on the display1730, and thus enjoys playing the dance game by moving the entire body.

And further, as the player 1800 dances, the player's movements aredetected by the artificial retina camera 1720, and are shown by themovements of the player character 1960 presented in a window 1958 on thescreen. The player 1800 can thus enjoy the dance game by observing themovements of the player character 1960 to confirm the player's owndancing form. The player 1800 can thus enjoy experience using the wholebody to play the dance game.

And further, the dancing movements of the instructor character 1956 andthe dancing movements of the player 1800 detected by the artificialretina camera 1720 can also be compared by the game apparatus 1700 toevaluate the similarity between the instructor's form and the player'sown form, thus providing yet another way for the player 1800 to enjoythe dance game.

When the dance game is over, a screen is presented on display 1730enabling the player 1800 to enter certain player's information (forexample, such as the player's initials or name) using the operatingbuttons 1710 (step S107). The information of the player 1800 and thescore are then shown on the display 1730, and the game ends.

FIG. 8 is a functional block diagram of the game apparatus 1700according to the first embodiment of the invention. The functional blockcomprises an input operating unit 1100, a movement detection unit 1200,a processing unit 1000, a display unit 1300, a speaker 1400, and a datastorage medium 1500.

The input operating unit 1100 is a means enabling the player 1800 toenter an operating data using the operating buttons 1710 shown in FIG. 1or the foot-controlled buttons or the touch-panel type buttons as notedabove. The operating data obtained through the input operating unit 1100is outputted to the processing unit 1000.

The movement detecting unit 1200 is achieved as a function of theartificial retina camera 1720 shown in FIG. 1 and detects the player'smovement. The movement detecting unit 1200 thus comprises aphoto-detection unit 1220, a photo-detection control unit 1240, and animage analyzing unit 1260.

The photo-detection unit 1220 comprises a plurality of thephoto-detector elements arranged in a two-dimensional array (a latticeshape). The photo-detection control unit 1240 controls thephoto-detection sensitivity of each of the photo-detector elements.Because the photo-detection control unit 1240 controls thephoto-detection sensitivity of each of the photo-detector elements, aparticular image process such as edge detection process for detectingonly the contours of the captured image, is applied to the image data asit is captured and photo-detected by the photo-detection unit 1220. Theprocessed image thus obtained by the photo-detection unit 1220 is thenoutput to the image analyzing unit 1260 as a processed image signal.

The image analyzing unit 1260 then detects the two-dimensional movementin the processed image signal inputted from the photo-detection unit1220 by applying an optical flow technique or other process to eachprocessed image signal. That is, the direction, the strength, and themagnitude of the movement in each pixel are detected from the temporaland spatial change in the brightness of each pixel. The image analyzingunit 1260 then identifies areas of the strong movement from the detectedoptical flow, and the direction and the strength of the two-dimensionalmovement is calculated for the overall region. More specifically, thetwo-dimensional movement is obtained using the vector average of theoptical flow in each area.

FIG. 9 shows an example of detection results for a particular movementof the player 1800, specifically a movement of one arm. The arrows shownin FIG. 9 are vectors of the movement of the arm of the player 1800, andindicate how the image signal for the arm changed.

As noted above, the movement detection unit 1200 constantly detects themovement of a particular object, which is primarily the player in thisembodiment. The movement detection unit 1200 then outputs the detectedplayer's movement as the input data to the processing unit 1000, andoutputs the player's image data to the processing unit 1000.

The movement detection unit 1200 thus both captures a player's image andprocesses the captured image. The movement detection unit 1200 alsodetects the player's movements from the processed image. That is, theprocessing capacity of the processing unit 1000 can be directed to otherprocesses because it does not perform the image processing and themotion detection, and the overall processing capacity of the processingunit 1000 can thus be improved.

The movements of the player 1800 detected by the movement detection unit1200 are then outputted to the processing unit 1000 as an input datasuch as shown in FIG. 10. FIG. 10 shows an example of an input data ofthe player 1800 outputted at a particular timing from the movementdetection unit 1200 (the image analyzing unit 1260) to the processingunit 1000. Note that this input data is input continuously to theprocessing unit 1000, and that data for one movement of the player 1800is shown in FIG. 10. If the player moves both arms, for example, twodata sets such as shown in FIG. 10 are outputted from the movementdetection unit 1200, and the number of data sets and the content of eachchanges dynamically according to the player's movement. As shown in FIG.10, the input data for the player 1800 is a vector data comprising adirection α1, a magnitude β1, and a speed γ1.

Referring again to FIG. 8, the data storage medium 1500 stores data anda program for accomplishing various processes. This data storage medium1500 can be a hardware device such as an IC card, a magneto-optical disk(MO), a floppy disk (FD), DVD, a hard disk, a semiconductor memory, orthe like. The data storage medium 1500 in this preferred embodimentcomprises a game program 1510, an assignment data 1530, a character data1520, and a movement input program 1540.

The game program 1510 is a program for handling a process fordetermining the progress of the game, a process for calculating andtallying the score of the game, a process for setting object spaces, andother processes for playing the game.

The assignment data 1530 comprises a sound data 1532 for generating asound, a music, and sound effects (collectively referred to below as theaudio data), that is, for setting an assigned dance and a sound, amusic, and sound effects during dancing; a character movement data 1534for setting a movement of a character; and a decision reference data1536 for deciding the similarly between the assigned dance and theplayer's dance.

The character data 1520 defines one or a plurality of differentcharacters, as well as such related information as feature and posturedata (key frames) for each character.

The movement input program 1540 controls various processes related tothe movement detection unit 1200, including a process for detecting amovement of a particular object by means of the movement detection unit1200 and then inputting this movement as an input data to the processingunit 1000, and a process for handling data input/output between themovement detection unit 1200 and the processing unit 1000.

The decision reference data 1536 is a data used in a process for settingthe assigned dance by means of an assigned dance setting unit 1014, anda process for comparing the movement of the player 1800 and the assigneddance by a similarity decision unit 1016.

FIG. 11 shows the configuration and an example of the decision referencedata 1536. Types of dance are stored with a specific correlation to thedecision reference data at the timing appropriate to the type of dance.Various parts of the body and the decision reference data for each bodypart are also stored with a correlation therebetween. The decisionreference data comprises a direction α, a magnitude β, and a speed γ.For example, the decision reference data 1536 at a time t1 for a bodypart “a” in dance 1 are a direction all, a magnitude β11, and a speedγ11.

The character movement data 1534 is data used in the process whereby thegame operation unit 1010 causes the instructor character 1956 to performthe assigned dance.

FIG. 12 shows the configuration and an example of the character movementdata 1534 in dance 1. The character movement data stores various bodyparts (such as the character's hands, feet, face, arms, legs, and chest)correlated to the position of each part in a particular key frame foreach particular dance. For example, a part “a” of a character A in dance1 is at a position (Xa1, Ya1, Za1) in a frame 1, and is at a position(Xb1, Yb1, Zb1) in a frame 2. As shown in FIG. 12, the data table isstored in the data storage medium 1500, according to the type of thedance.

Referring again to FIG. 8, the processing unit 1000 inputs data to andreceives data from the input operating unit 1100, the movement detectionunit 1200, the display unit 1300, the speaker 1400, and the data storagemedium 1500. The processing unit 1000 also runs various processes basedon the detection data (the image data and the input data) from themovement detection unit 1200, the operating data from the inputoperating unit 1100, and various data and various programs stored to thedata storage medium 1500, and outputs the data to the display unit 1300and the speaker 1400. The processing unit 1000 also controls the datainput/output with the movement detection unit 1200 based on the movementinput program 1540.

The functions of this processing unit 1000 can be achieved using suchhardware as a CISC or RISC CPU, a DSP, an image capture IC, a clockcircuit, and so on.

The display unit 1300 presents a game screen according to the data fromthe processing unit 1000.

The speaker 1400 also generates the game sounds according to the datafrom the processing unit 1000.

The processing unit 1000 also comprises a game operation unit 1010, animage generation unit 1030, and a sound generation unit 1040.

The game operation unit 1010 runs processes for: advancing a game playbased on the game program 1510; setting object spaces based on the gameprogram 1510; setting objects in the object spaces based on the gameprogram 1510; moving the instructor character 1956 (that is, making theinstructor character 1956 dance) based on the character movement data1534, for example; moving the player character 1960 (that is, making theplayer character 1960 dance) based on data of the player 1800(particularly the above-noted input data) detected by the movementdetection unit 1200; and setting the game level (level of difficulty) ofthe assigned dance.

The process for operating the instructor character 1956 is describednext.

The instructor character 1956 is made to dance based primarily on thecharacter movement data 1534. The game operation unit 1010 creates thecharacter movement by interpolating a value indicative of the change inthe position of the character in one frame of the character movementdata 1534 at a particular time to the position of the character in thenext frame. By linear interpolation of the change in character positionfrom the frame 1 to the frame 2, the game operation unit 1010 can createan animated character that moves continuously from the frame 1 to theframe 2, for example.

The process for operating the player instructor character 1960 isdescribed next.

The player character 1960 is also moved based on the character data 1520and the above-noted input data inputted from the movement detection unit1200. Because the input data relates to a two-dimensional movement, themovement of the player character 1960 is expressed as the movementdetected by the movement detection unit 1200, that is, the playermovement as seen from the front. It will be obvious to one with ordinaryskill in the related art that an image of the player's movement can alsobe generated from the above-noted image data.

The game operation unit 1010 also contains a character setting unit1012, an assigned dance setting unit 1014, a similarity decision unit1016, a sound setting unit 1018, a tempo adjusting unit 1020, and a gamelevel setting unit 1022.

The character setting unit 1012 runs a process for setting theinstructor character 1956 and the player character 1960 based on thecharacter data 1520 and the data inputted by means of the inputoperating unit 1100 or the movement detection unit 1200.

The game level setting unit 1022 runs a process for setting the gamelevel (level of difficulty) based on the data inputted by means of theinput operating unit 1100 or the movement detection unit 1200.

The assigned dance setting unit 1014 runs a process for setting theassigned dance based on the decision reference data 1536, and settingthe type (difficulty) of the assigned dance based on the game level setby the game level setting unit 1022.

The sound setting unit 1018 runs a process for setting the sound fromthe sound data 1532 based on the data inputted by means of the inputoperating unit 1100 or the movement detection unit 1200.

A tempo adjusting unit 1020 adjusts the tempo of the assigned dance setby the assigned dance setting unit 1014 based on the game level ofdifficulty set by the game level setting unit 1022 or the game operationunit 1010. The tempo adjusting unit 1020 also sets the tempo of themusic set by the sound setting unit 1018 based on the game level ofdifficulty set by the game level setting unit 1022 or the game operationunit 1010. The tempo adjusting unit 1020 can thus be used to adjust thedifficulty of the dance game or make the dance game more interesting by,for example, matching the tempo of the music to the assigned dance,changing the tempo according to the progress of the game, or making thetempo of the music and the assigned dance different.

The process of the similarity decision unit 1016 is described asfollows. The similarity decision unit 1016 compares the decision data ofthe decision reference data 1536 at a particular time and the input dataof the player's movement (shown in FIG. 10), which is continuouslydetected by the movement detection unit 1200, at the same time todetermine the similarity therebetween. For example, if the input datafor player body part “a” at the time t1 is the direction α1, themagnitude β1, and the speed γ1 as shown in FIG. 10, this player'smovement data is compared with the decision reference data 1536 at thesame time t1, that is, the direction α11, the magnitude β11, and thespeed γ11. As the difference between the player's movement data and thedecision reference data 1536 decreases, the similarity therebetweenincreases; as the difference increases, the similarity decreases. Thissimilarity comparison and decision is repeated at each timing interval,that is, at the times t1, t2, t3, and so forth. It will be obvious toone with ordinary skill in the related art that the similaritycomparison and decision is not required at all timing points t1, t2, t3,and so forth, and can be performed at the selecting timing points, suchas t1 and t3.

Further, the similarity decision unit 1016 can also adjust the decisionresults based on the game level set by the game level setting unit 1022.That is, the input data for the same movement of the player 1800 can bescored lower if the game level is set to a high level of difficulty, andhigher if the game level is low.

Furthermore, when the tempo adjusting unit 1020 adjusts the tempo of theassigned dance, the tempo adjusting unit 1020 adjusts the tempo of theassigned dance by adjusting the timing interval according to the tempoof the music, while the similarity decision unit 1016 also adjusts thetiming interval of deciding the similarity according to the tempo of themusic.

Referring again to FIG. 8, the game operation unit 1010 performs ascoring process based on the game program 1510 and the decision returnedby the similarity decision unit 1016. Further, the game operation unit1010 determines the game progress based on the scoring process (that is,whether to allow the game to continue or to end the game).

The image generation unit 1030 also comprises a game image generationunit 1032 and a player image generation unit 1034.

The game image generation unit 1032 runs a process for generating animage of the instructor character 1956 dancing in the object space setby the game operation unit 1010 as seen from a particular view point(referred to below as a virtual camera), and a process for generating animage of the objects set in the object space by the game operation unit1010. Note that game image generation unit 1032 can be comprised tochange the view point by, for example, rotating the virtual camera inthe object space or changing the range of the viewing field. The gameimage generation unit 1032 can thus generate images in which theinstructor character 1956 turns or moves closer to or away from thevirtual camera.

The game image generation unit 1032 also runs a process for generatingan image for displaying the timing for deciding the similarity by thesimilarity decision unit 1016, a process for generating an image fordisplaying the tempo adjusted by the tempo adjusting unit 1020, and aprocess for generating an image for displaying the results of thescoring process performed by the game operation unit 1010.

The player image generation unit 1034 runs a process for generating animage of the player character 1960 moving (dancing) as determined by thegame operation unit 1010, or an image data (a moving image) of theplayer 1800 detected by the movement detection unit 1200. The imagesgenerated by the player image generation unit 1034 are presented on thedisplay unit 1300.

The sound generation unit 1040 runs a process for generating the sound(the sound wherein the tempo thereof is adjusted by the tempo adjustingunit 1020) set by means of the sound setting unit 1018, and a processfor generating an alarm sound notifying the player of the timing atwhich the similarity decision unit 1016 decides the similarity. Thegenerated sounds are outputted from the speaker 1400.

The game apparatus 1700 is comprised as described above, according tothe first embodiment of the present invention. Playing a game in thegame apparatus 1700 will be explained, as follows. FIG. 13 is a flowchart showing the flow in the operation of this game apparatus 1700while a game is being played.

The game operation is started on the basis of the data input from theinput operating unit 1100 and the movement detection unit 1200, and thedata and the program stored on the data storage medium 1500 (step S111).

That is, the assigned dance setting unit 1014 sets the assigned danceand the sound setting unit 1018 sets the sound. Note that as the gamelevel set by the game level setting unit 1022 increases, the assigneddance is set to one requiring a more difficult movement or a fastermovement. Further, as the game level set by the game level setting unit1022 increases, the similarity decision unit 1016 is set to lower thesimilarity ratings.

The tempo adjusting unit 1020 also adjusts the tempo of the assigneddance and the tempo of the music. For example, as the game level set bythe game level setting unit 1022 increases, a faster tempo is set.

The game play is started when these settings of the game operation arecompleted (step S112).

Once the game play is started, a memorizing mode is started (step S113).The memorizing mode lets the player remember the assigned dance. Thatis, according to the memorizing mode, the instructor character 1956dances the assigned dance so that the player can memorize the assigneddance. Further, in the memorizing mode, the music set by the soundsetting unit 1018 is outputted from the speaker 1400 at the tempo set bythe tempo adjusting unit 1020. And further, the dance of the instructorcharacter 1956, which is controlled by the game operation unit 1010, isshown on the display unit 1300 dancing rhythmically to the music duringthe memorizing mode.

In the memorizing mode, the movements of the player 1800 are alsodetected by the movement detection unit 1200 and shown on the displayunit 1300. The movements of the player 1800 are described by displayingthe movements of player character 1960. Note that the player character1960 can be displayed as a projection (a dot image, for example) of theplayer 1800.

The dance mode is started after the memorizing mode (step S114). Thedance mode is the mode in which the player's movement (dancing) iscompared with the assigned dance and decided. The music used in thememorizing mode is also outputted in the dance mode. The instructorcharacter 1956 also dances the same dance performed in the memorizingmode.

The similarity decision unit 1016 also begins comparing the similaritybetween the assigned dance and the dance performed by the player 1800.That is, as noted above, the decision data from the decision referencedata 1536 at a particular timing is compared with the input data of theplayer 1800 outputted from the movement detection unit 1200 at the sametiming, to decide the similarity therebetween, as described above. Thegame operation unit 1010 then processes the score of the player 1800 onthe basis of the result of the comparison and decision.

The game apparatus 1700 according to the first embodiment progresses thegame by repeating the memorizing mode and the dance mode (step S115;NO). A particular stage of the game ends after these steps are repeateda specified number of times (step S115; YES).

If the resulting of processing the score by the game operation unit 1010satisfies a particular condition, the game advances to the next stage;if these conditions are not satisfied, the game is over. The gameoperation unit 1010 determines of these conditions are satisfied. If thegame advances to the next stage, the tempo of the assigned dance can beincreased or the type of the assigned dance can be changed, to raise theassigned dance's level of difficulty. Further, if the game advances tothe next stage, the dance of the instructor character 1956 in the dancemode can be changed so as to be different from the assigned dance ofinstructor character 1956 in the memorizing mode, to raise the gamelevel of difficulty. By thus raising the level of difficulty, the gamecan be enjoyed even by advanced players who have become accustomed tothe game.

FIG. 14 shows an exemplary display screen in the memorizing mode, andFIG. 15 shows an exemplary display screen in the dance mode.

A feature of the first embodiment of the invention is that, as shown inFIG. 14 and FIG. 15, the movement of the player 1800 is shown in awindow 1958 at the bottom right of the screen in both the memorizingmode and the dance mode.

That is, the movement detection unit 1200 detects the movement of theplayer 1800, and outputs the detected movement as an input data (avector data) to the processing unit 1000. When the processing unit 1000receives this input data, the game operation unit 1010 displays themovement of the player 1800 based on this input data by some means. Theplayer 1800 can then check the player's form shown on the screen whileplaying the game. Therefore, the player 1800 can enjoy the game bywatching the player's own dance form on the screen. Further, the player1800 can compare the player's own movements shown on the screen with themovement of the instructor character 1956, and thereby correct theplayer's own form. Accordingly, the player 1800 is able to improve thesimilarity between the player's dance form and the assigned dance formto improve more quickly.

According to the first embodiment, the movement of the player 1800 isindicated as the movement of the player character 1960.

That is, the game operation unit 1010 sets the movement of the playercharacter 1960 and displays the movement of the player character 1960 onthe screen on the basis of the input data outputted to the processingunit 1000 from the movement detection unit 1200. Because the movement ofthe player 1800 is displayed as the movement of the player character1960, the player 1800 can immediately check the player's form. That is,by displaying the movement of the player character 1960, the player 1800can immediately confirm the player's form. Further, because the movementof the player 1800 is displayed as the movement of the player character1960, the player 1800 can confirm the player's form with gettinginterested. Note, further, the movement of the player 1800 can bedisplayed by simply presenting the image data detected by the movementdetection unit 1200 directly on the screen, that is, the movement of theplayer 1800 can be recorded for direct presentation without generatingan animated character.

According to the first embodiment, the music is also outputted from thespeaker in both the memorizing mode and the dance mode.

That is, the music selected from the sound data 1532 by the soundsetting unit 1018 is outputted from the speaker 1400. The player 1800can therefore play the dance game while listening to the selected music.The player 1800 can also dance to the rhythm and the tempo of theoutputted music, and can thus dance rhythmically. Furthermore, if theplayer 1800 is dancing to the music, other people can enjoy watching thedance of the player 1800 and thus be motivated to try the dance gamethemselves.

The tempo of the assigned dance is also matched to the tempo of themusic outputted, by the tempo adjusting unit 1020.

By thus matching the tempo of the dance and the assigned music, theplayer can move according to the assigned dance. Even beginners cantherefore enjoy the dance game.

It should be noted that, for example, when the game level of difficultyset by the game level setting unit 1022 is high, it is also possible tovary the tempo of the assigned dance from the tempo of the music. Thiscan be accomplished that the tempo of the music set by the sound settingunit 1018 is not matched with the tempo of the assigned dance byadjusting the tempo of the music, by the tempo adjusting unit 1020.

Because the tempo of the assigned dance and the tempo of the music donot match with each other, the player 1800 cannot use the tempo of themusic to determine the tempo of the assigned dance. Moving according tothe assigned dance can thus be made more difficult even for the advancedplayers that have played the game continuously or frequently and haveremembered the different assigned dances. Even the advanced players willthus not grow tired of the dance game.

According to the first embodiment of the present invention, as shown inFIG. 14 and FIG. 15, the tempo of the assigned dance is displayed on atiming gauge 1902. The timing gauge 1902 is composed so that the needle1904 spins at a speed determined by the tempo. The timing gauge 1902 isfor displaying the tempo of the assigned dance tempo adjusted by thetempo adjusting unit 1020, and is generated for presentation on thescreen by the game image generation unit 1032.

Because the tempo of the assigned dance is displayed on the timing gauge1902, the player is able to move according to the tempo of the assigneddance. Even the beginners can therefore enjoy playing the dance game.Furthermore, even if the tempo of the music and tempo of the assigneddance differ, the player 1800 can watch the timing gauge 1902 todetermine the tempo of the assigned dance.

The timing at which the similarity decision unit 1016 determines thesimilarity can also be shown in the timing gauge 1902. For example, thecolor of the panel part 1906 of the timing gauge 1902 could change whenthe similarity is decided. The timing at which the needle 1904 forspinning overlaps the scale 1908 of the timing gauge 1902 could also beadjusted to matching the timing at which the similarity is decided. Thebackground color of the screen could also be changed according to thetiming at which the similarity is decided.

Because the similarity decision unit 1016 displays the timing fordeciding the similarity on the screen, the player 1800 can confirm thetiming for deciding the similarity by visual effects. The player 1800 isalso better able to move at the timing for deciding the similarity, andcan thus dance rhythmically. Accordingly, the player can enjoy playing amore interesting dance game.

It will also be obvious that audible cues, such as voicing “1, 2, 3”,can also be outputted to the timing for deciding the similarity so thatthe player can recognize the timing for deciding the similarity by suchaudible cues. To accomplish this, the sound generation unit 1040produces the voice outputted at the timing for deciding the similarity.It will also be obvious that a sound effect, such as a rhythmical beat,can be used instead of a voice.

According to the first embodiment, the score 1910 is also displayed atthe top of the screen as shown in FIG. 14 and FIG. 15. The score 1910 istypically the cumulative score since the game play started. As describedabove, the game operation unit 1010 calculates the score based on thesimilarity result outputted by the similarity decision unit 1016 at eachtime. This score is calculated as, for example, the score relative tothe maximum possible score at the time the similarity was decided. Thegame operation unit 1010 accumulates the number of scores scored at eachtime, and the game image generation unit 1032 generates an image of thecumulative score. The score 1910 is thus displayed.

Because the total number of scores accumulated since the start of thegame play is shown in the score 1910, the player 1800 can decide howclosely the player's movements have cumulatively resembled the assigneddance since the game started. Accordingly, the player 1800 can bemotivated to score as many points as possible, and the player 1800 canplay the game with the desire to improve as much as possible.

Other people watching the score 1910 might also feel they can score evenhigher, and can thus also be motivated to play the game and do evenbetter.

According to the first embodiment, the result of deciding the similarityis also displayed on the right side of the screen in the dance mode asshown in FIG. 15. In FIG. 15, for example, “Excellent” is displayed as adecision result 1912, and the result of deciding the similarity betweenthe movement of the assigned dance at a particular time and the movementof the player 1800 at the same time is thus displayed.

In the first embodiment of the present invention, the similarity isdecided and displayed at four levels. When “Excellent” is displayed theplayer's movements are nearly identical to the assigned dance whendecided. As the decision result 1912, “Good” is displayed when thesimilarity is better than average but not excellent, “Poor” is displayedwhen there is little resemblance, and “Failure” is displayed when thereis absolutely no similarity between the movements.

One method for calculating the decision is described by way of exampleonly below as it will be obvious that other methods can alternatively beused.

According to the first embodiment, the decision result 1912 is displayedbased on the result of deciding the similarity by the similaritydecision unit 1016. That is, the game operation unit 1010 calculates thescore based on the similarity decision unit 1016, and the game operationunit 1010 selects a decision according to the score.

For example, if the four possible ranges out of a maximum 100 points are0 to 25, 26 to 50, 51 to 75, and 76 to 100 points, “Excellent” isselected when the calculated score is 76 to 100 points, “Good” isselected when the calculated score is 51 to 75 points, “Poor” isselected when the calculated score is 26 to 50 points, and “Failure” isselected when the calculated score is 0 to 25 points. The game imagegeneration unit 1032 then generates an image of the selected decision.

Further, according to the first embodiment, it will be obvious to onewith ordinary skill in the related art that while the four evaluationlevels are used, some other plurality of levels can be alternativelyused. In addition, a numerical evaluation, such as 1, 2, 3, and 4, orsome set of symbols, such as “∘”, “Δ”, and “X”, could be displayed inplace of letters or words.

The player 1800 is therefore able to determine the correctness of theplayer's movements at the decision time because a decision according tothe similarity at a particular timing for deciding the similarity isdisplayed by means of the decision result 1912. The player 1800 can thusconfirm the comparison, and play the dance game with the determinationto improve the decision while continuing to dance.

A level gauge 1914 is also displayed on the left side of the screen inthe first embodiment, as shown in FIG. 14 and FIG. 15. This level gauge1914 moves up and down on the screen according to the similaritydecision, and thus fluctuates according the result of comparing themovement of the assigned dance at a particular time with the movement ofthe players 1800 at that same time. That is, the level of the levelgauge 1914 rises as the resemblance between the player's movements andthe movement of the assigned dance increases, and descends and theresemblance decreases.

It will be obvious that the operation of the level gauge 914 shall notbe limited the above-noted example.

As with the above-described decision calculation, the game operationunit 1010 runs a process for calculating the score and changing thelevel gauge based on the calculated score. The game image generationunit 1032 generates an image based on the process. For example, if thegame operation unit 1010 determines an “Excellent” decision, the levelgauge 1914 is raised two bars; if the game operation unit 1010determines a “Good” decision, the level gauge 914 is raised one bar; ifthe game operation unit 1010 determines a “Poor” decision, the levelgauge 1914 is lowered one bar; and if the game operation unit 1010determines a “Failure” decision, the level gauge 1914 is lowered twobars.

Alternatively, the level gauge 1914 can be raised if the scorecalculated by the game operation unit 1010 is higher than a certainvalue, and the level gauge 1914 can be lowered if the score calculatedby the game operation unit 1010 is lower.

Because the level gauge 1914 thus fluctuates according to the similaritybetween the player's movements and the movement of the assigned dance ata particular time, the player 1800 can dance while comparing theplayer's movements with the assigned dance. This game apparatus 1700 canthus encourage the player 1800 to improve, and the player can play thegame in earnest, using it as a learning aid.

The tempo of the assigned dance can also be set according to thefluctuation of the level gauge 1914. For example, as the level gauge1914 drops and approaches the bottom, the tempo of the assigned dancecould be slowed. That is, because the level gauge 1914 moves up and downbased on the decision result returned by the similarity decision unit1016, the tempo of the assigned dance can also be changed based on thedecision result by the similarity decision unit 1016. Yet further, thegame could be ended if the level gauge 1914 reaches the bottom.

The type of assigned dance can also be changed according to the changein the level gauge 1914. For example, if the level gauge 1914 exceeds aparticular level, the assigned dance set in the next memorizing mode orthe next dance mode could be set to a more complicated movement or afaster movement, and while if the level gauge 1914 drops below aparticular level, the assigned dance could be set to an easier dance. Inother words, because the level gauge 1914 moves up an down based on thedecision result from the similarity decision unit 1016, the type ofassigned dance can also be changed based on the decision result from thesimilarity decision unit 1016.

The decision results returned by the similarity decision unit 1016 canalso be changed according to the change in the level gauge 1914. Forexample, if the level gauge 1914 drops below a certain level, assumingplayer 1800 movement is the same, the similarity decision unit 1016decides that the same movement of the player 1800 has a highersimilarity, and if the level gauge 1914 is above a certain level, thesimilarity decision unit 1016 decides that the same movement of theplayer 1800 has a lower similarity. In other words, because the levelgauge 1914 moves up and down based on the result from the similaritydecision unit 1016, the type of the assigned dance also changes based onthe decision result from the similarity decision unit 1016.

As described above, the first embodiment of the present invention hastwo modes, a memorizing mode and dance mode.

According to the memorizing mode, as shown in FIG. 14, “Memorize” isdisplayed on the right side of the screen. When the player 1800 sees“Memorize” displayed, the player knows that the game apparatus is in thememorizing mode.

Further, “Memorize” is also displayed flashing to match the tempo of theassigned dance. As the tempo of the assigned dance increases, “Memorize”flashes more quickly; as the tempo of the assigned dance slows,“Memorize” flashes more slowly. The player 1800 thus knows the tempo ofthe assigned dance in the memorizing mode from the speed of the flashingindicator.

It should be noted that the instructor character 1956 continues dancingand the music does not stop when changing from the memorizing mode tothe dance mode and from the dance mode to the memorizing mode in thefirst embodiment of the invention. Furthermore, if the player 1800 doesnot recognize the moment at which the mode changes, the player 1800 willnot know when to start dancing.

Therefore, as shown in FIG. 16, a dance mode start notice 1916 isdisplayed in the top left of the screen when in the memorizing mode toinform the player 1800 when to start dancing. By displaying “Now yourturn!!” as shown in FIG. 16, the player 1800 is visually cued to thestart of the dance mode. That is, the player 1800 is notified that thecomparison of the movements of the player 1800 with the assigned danceby the similarity decision unit 1016 is to begin. The dance mode startnotice 1916 is displayed immediately before the dance mode starts, thatis, immediately before the similarity decision by the similaritydecision unit 1016 starts. The game operation unit 1010 sets the displayof the dance mode start notice 1916, and the game image generation unit1032 generates the image of the dance mode start notice 1916.

It will be obvious to one with ordinary skill in the related art thatrather than displaying a visual cue, the player can be notified by anannouncement from the speaker 1400 that “Now your turn!!”. The player1800 can thus also be audibly notified that the dance mode is starting.

By thus displaying the dance mode start notice 1916 immediately beforethe start of the dance mode, the player 1800 can easily start dancing inthe dance mode.

Like changing from the memorizing mode to the dance mode, the player1800 can be notified before changing from the dance mode to thememorizing mode that the memorizing mode is to start. FIG. 17 shows anexample of a notice informing the player that the memorizing mode is tostart. As shown in FIG. 17, the memorizing mode start notice 1918 ispresented in the top right area of the screen with the words “Followme!!” in this example.

It will also be obvious that an audible “follow me” notice could beoutputted from the speaker 1400.

By thus displaying the memorizing mode start notice 1918 in the dancemode before changing to the memorizing mode, n the player 1800 caneasily know that the memorizing mode is starting.

As described above, in the first embodiment of the present invention,the dance mode starts after the memorizing mode. That is, the assigneddance is shown to the player 1800 by the instructor character 1956dancing in the memorizing mode, and then the movements of the player'sdance are decided using the same dance in the dance mode. Therefore, theplayer 1800 can learn the assigned dance in the memorizing mode firstbefore attempting to perform the same dance in the dance mode fordecision. Accordingly, the dance game avoids becoming a reflexive gamerequiring a simple reflexive reaction to displayed movements, and theplayers can have fun playing the dance game.

It is also possible to not display the movements of the instructorcharacter 1956 or show the assigned dance in the memorizing mode and thedance mode. By thus not showing the assigned dance on the screen, it isdifficult for even advanced players that have become accustomed to thegame to perform the assigned dance. Even advanced players can thereforebe motivated to enjoy playing the game.

An exemplary hardware configuration of the game apparatus 1700 accordingto the first embodiment is described next below with reference to FIG.18. The game apparatus 1700 shown in FIG. 18 comprises a CPU 1600, a ROM1602, a RAM 1604, a data storage medium 1606, a sound generation IC1608, an image generation IC 1610, and I/O ports 1612, 1614, and 1626,wherein they are interconnected by a system bus 1616 so that data can beexchanged therebetween. A display 1618 is further connected to the imagegeneration IC 1610, a speaker 1620 is connected to the sound generationIC 1608, a control device 1622 is connected to the I/O port 1612, acommunication device 1624 is connected to the I/O port 1614, and amovement detecting device 1628 is connected to the I/O port 1626.

The data storage medium 1606 stores primarily a software program, a datafor expressing displayed objects, and a game data. According to aconsumer game machine, for example, a CD-ROM, a game cartridge, a DVD,or other medium is typically used as the data storage medium for storingthe game program and other data, and a memory card or other medium isused as the data storage medium for storing the game (playing) data.According to an arcade game machine according to the first embodiment ofthe invention, such a semiconductor memory device as a ROM or othermedium or a hard disk is used, in which case the data storage medium1606 can be achieved in the ROM 1602. It should be noted that in thefirst embodiment, the game program 1510, the character data 1520, theassignment data 1530, the sound data 1532, the character movement data1534, the decision reference data 1536, and the movement input program1540 are stored to the memory.

The control device 1622 is equivalent to the operating buttons, the gamecontroller, the operating panel, or other such device, and is the deviceused by the user to input the results of the decisions received whileplaying the game to the game apparatus. According to the firstembodiment, the operating buttons 1710 are equivalent to the controldevice 1622.

The movement detecting device 1628 detects the player's movement and theimages, and outputs the detection data. The artificial retina camera1720 shown in FIG. 1 is equivalent to the movement detecting device1628. It should be noted that a processing device separate from the CCDcamera can be used to perform the edge detection or other image process,and detect two-dimensional movements by applying an optical flowtechnique to the processed image. In other words, a device other thanthe CPU 1600 accomplishes the image processing and the movementdetection, and the processing capacity of the CPU 1600 will thereforenot drop.

The CPU 1600 provides overall control of the game apparatus and variousdata processing operations according to signals inputted by the controldevice 1622, the operating system (including initialization data for thegame apparatus) stored to the ROM 1602, and the game program stored tothe data storage medium 1606.

The RAM 1604 is a storage means used as a operating memory by the CPU1600, and stores particular contents from the data storage medium 1606or the ROM 1602, or operating results from the CPU 1600.

The sound generation IC 1608 and the image generation IC 1610 are alsodisposed in this type of game apparatus to generate and output soundsand images appropriate to the game.

The sound generation IC 1608 is an integrated circuit for generatinggame sounds, such as sound effects, background music and so on, based onthe data stored to the data storage medium 1606 or the ROM 1602, and thegenerated sounds are outputted by the speaker 1620.

The image generation IC 1610 is an integrated circuit for generatingpixel data to be outputted to the display 1618 based on the image datareceived, for example, from the RAM 1604, the ROM 1602, the data storagemedium 1606, and so on.

The display 1618 must have a light-emitting capability, and is typicallya CRT, a LCD, a plasma display, or other such display device.

The communication device 1624 is for communicating various data used bythe game apparatus with an external device. If the game apparatus isconnected with another game apparatus, the communication device 1624handles communicating data with the other game apparatus. Thecommunication device 1624 could also be used to communicate gameprograms and data with other devices via a communications line.

A process for detecting the movement, a process for calculating thescore, and a process for moving the animated characters are achieved bythe data storage medium 1606 which stores the game program 1510, themovement input program 1540, the character data 1520 and the assignmentdata 1530, the CPU 1600 which operates according to the program, theimage generation IC 1610, and the sound generation IC 1608.

It should be noted that the processes accomplished by the imagegeneration IC 1610 and the sound generation IC 1608 can be written assoftware applications run by the CPU 1600, a general purpose DSP, orother such device.

As described above, the game apparatus of the present invention, shownin FIG. 1 and FIG. 2 is an arcade game machine.

In such a case as described above, the CPU, the image generator IC, thesound generator IC, and other components are mounted on a system boardbuilt in to the game apparatus. The game program 1510, the assignmentdata 1530, and other data related to the dance game shown in FIG. 8 isalso stored in the memory as the data storage medium on the systemboard. The data related to the dance game is referred to below as the“stored data”.

FIG. 19A shows an example of the present invention applied to a computersystem.

In the computer system, the stored data is stored to such a data storagemedium capable of being installed in and removed from the computersystem, as a CD-ROM, a DVD, or the like. Therefore, the player watchesthe images presented on the display 1850 while dancing (moving) in frontof the artificial retina camera 1852 to enjoy playing the dance game.

FIG. 19B shows an example of the present invention applied to a gamesystem comprising a host apparatus 1860 and terminals 1864-1 to 1864-nconnected to the host apparatus 1860 by way of a communication line1862.

In the game system, the stored data is stored to such as a data storagemedium capable of being controlled by the host apparatus 1860, as, forexample, a magnetic disk device, a magnetic tape device, a memory, orthe like. When each of the terminals 1864-1 to 1864-n comprises a CPU,an image generator IC, a sound generator IC, and an artificial retinacamera and is capable of generating game images and game sounds(including background music) as a stand-alone device, the host apparatus1860 sends a game program for generating the game images and gamesounds, and so on to the terminals 1864-1 to 1864-n. On the other hand,when each of the terminals 1864-1 to 1864-n cannot generate game imagesand game sounds as a stand-alone device, the host apparatus 1860generates the game images and game sounds and sends them to each ofterminals 1864-1 to 1864-n to output them thereon.

An example of the game system is described next. FIG. 20 is an exemplarygame screen presented on the display of each terminal. That is, themovements of players who are playing the dance game at each terminal aredisplayed at the bottom of the game screen on each terminal.

In FIG. 20, the movements of the player 1 are shown by the playercharacter 1982 in the window 1970, the movements of the player 2 areshown as the player character 1984 in the window 1972, and the movementsof the player 3 are shown as the player character 1986 in the window1974. The cumulative score for the player 1 is also displayed in thescore 1976, the cumulative score for the player 2 is also displayed inthe score 1978, and the cumulative score for the player 3 is alsodisplayed in the score 980. The movement of each player playing thedance game is thus shown on each display. In other words, the movementsof other players playing the dance game are shown on one screen.

In such a game system as described above, each player learns theassigned dance by watching the instructor character 1956 performing theassigned dance in the memorizing mode, and the similarity between eachplayer's dance and the assigned dance is then decided in the dance mode,and the scores 1976, 1978, and 1980 change according to the result ofdeciding the similarity.

Each Player can thus enjoy comparing the own score with the otherplayers' scores. Furthermore, because the movements of all players areshown on one screen, each player can check the performances of the otherplayers while playing the game. Furthermore, each player can alsocompare the own form with that of the other players. Accordingly, thegame system enables plural players to play the dance game competingagainst other players.

As described above, it will thus be known from the above description ofthe first embodiment of the present invention that the movementdetection unit 1200 directly detects the movements of the player 1800and thus differs from a conventional video game apparatus requiring aplayer to hold flags in order to detect the player's movement. Becausethe game apparatus according to the invention thus directly detects themovement of the player's entire body, the players can enjoy theexperience of playing the game with their entire body. Further, becausethe player's movements (dance) is displayed while playing the game, theplayers can play the game while observing their own dance form on thescreen. Furthermore, because the music is also outputted while the gameis played, the players can also move (dance) rhythmically to the music.

It will be obvious to one with ordinary skill in the related art thatthe invention shall not be limited to the first embodiment describedabove, and can be varied in many ways without departing from the scopeof the following claims.

For example, the movement of the player 1800 is displayed in the windowat the bottom right of the screen on the display 1730 in the aboveembodiment, but a sub-monitor separated from the display 1730 could bedisposed on the back panel 1780 and the player's movement can bedisplayed on the sub-monitor. Other people could then watch the player'smovements on the sub-monitor. Further, the movements of the instructorcharacter 1956 could also be displayed on the sub-monitor.

Further, a light could also be disposed on the side of the dance stage1770 to illuminate the player 1800 from the side. Preferably, the lightemitted by the light is a linear light so that the back panel 1780 isnot illuminated. This prevents the player's shadow from being projectedonto the back panel 1780 so that the artificial retina camera 1720 willnot also detect the movement of the shadow.

Furthermore, one or a plurality of artificial retina cameras 1720 can beused. For example, the artificial retina cameras could be disposed tocapture images of the player from two different angles so that thethree-dimensional player's movements can be detected. That is, thedetection angles of the artificial retina cameras are not the same witheach other, and thereby the artificial retina cameras could detect thethree-dimensional player's movements including a depth. Accordingly, aneven more realistic dancing game can be provided by thus detecting theplayer's movements in the three dimensions.

Furthermore, the player character 1960 moves according to the input dataobtained by the artificial retina camera 1720 in the first embodiment,but the player character 1960 could alternatively move according to, forexample, the player's operation of a foot-operated buttons disposed onthe dance stage 1770 or a touch panel disposed near the dance positionof the player 1800. For example, the player could press the touch panelto make the player character 1960 turn its arm or play the tambourinesor other musical instrument.

Furthermore, the first embodiment of the present invention has beendescribed referring primarily to a single person playing the game, but aplurality of people could play the game at the same time. When aplurality of people play the game, a plurality of artificial retinacameras are disposed so that each camera could detect the movements ofone player. One artificial retina camera could alternatively be used todetect the movements of a plurality of players. Many variations of adance are possible with a plurality of players, and the game can thus bemade more interesting.

It will be obvious to one with ordinary skill in the related art thatwhile the first embodiment of the present invention is described abovewith reference to a dance game, the assigned movements other than thedance are also possible. For example, other possible actions used forthe game include movements for playing the drums or other musicalinstrument, pitching a ball, batting, copying an animal, caricaturingthe quirks of a famous person, and so on.

Effects according to the first embodiment of the present invention willbe explained, as follows.

As described above, according to the first embodiment of the presentinvention, the player moves according to the assigned movementsdisplayed on the assigned movement display unit, and thereby the game isplayed. Thereafter, the movement detecting unit then detects theplayer's movements, the similarity decision unit compares the player'smovements with the assigned movements, and the game operating unitprocesses the score on the basis of the decision result. Therefore, thegame progress is then determined according to the result of processingthe score. For example, if the result of processing the score isdisplayed, the player can enjoy comparing the player's movements withthe assigned movements. Accordingly, because the player moves accordingto the assigned movements displayed on the assigned movement displayunit, the player can enjoy experiencing playing the game using theentire body.

Further, the player can play the game while listening to music and soundeffects because the sound generation unit generates at least music,sound or sound effects. And further, the player can also move in timewith the music or the rhythm and the tempo of the music, and is thushelped to move rhythmically. Furthermore, if the player movesrhythmically to the sound, the music, or sound effects, other people canenjoy watching the player's perform and can be motivated to play thegame, too.

And further, the player can play the game while watching the player'sown movements detected by the movement detecting unit and displayed onthe player movement display unit. Accordingly, the player can enjoywatching the player's own movements. The player can also confirm whetherthe rhythm and the tempo of the player's movements match the rhythm andthe tempo of the sound or the music generated by the sound generationunit.

Furthermore, the player's movements can be detected without acomplicated image processing operation because the player's movementsare detected by the movement detecting unit. That is, unlike aconventional video game apparatus in which the player must hold flags,for example, in order to detect the player's movements, the gameapparatus according to the present invention detects movement of theentire player's body, and the player can therefore enjoy playing thegame using the whole body.

Second Embodiment

The game apparatus 2700 which can progress a game and process a scoreaccording to a movement using an entire body without restricting aninputting action, will be explained according to the second embodimentof the game apparatus of the present invention, with reference to FIGS.1, 2 and 21 to 36.

A front view showing an external of a housing 2750 of the game apparatus2700 as an arcade game machine according to the second embodiment of thegame apparatus of the present invention, and a vertical sectional viewof the ill game apparatus 2700 showing a positional relation between aplayer 1800 and an image sensor 2720 and a display 2730 which aredisposed in the housing 2750, are the same as the front view and thevertical sectional view of the game apparatus 1700 according to thefirst embodiment, as shown in FIG. 1 and FIG. 2. Therefore, the views isnot omitted to show in figures.

The game apparatus 2700 comprises: an image sensor 2720 (for example, anartificial retina camera) for detecting movements of the player 1800; adisplay 2730 for displaying a game screen; operating buttons 2710 forinputting an instruction to start a game or the like; a speaker 2760 foroutputting a BGM of a dance and voices for taking rhythms; and a light2790 for illuminations.

The game apparatus 2700 is one for progressing the dance game andcalculating the score by deciding whether or not the movements of theplayer 1800, detected by the image sensor 2720 are identical to themovements of the dance assigned (as will be called the “assigneddance”).

Further, the game apparatus 2700 progresses the game by repeating theexecutions of two kinds of modes of; a memorizing mode for making theplayer memorize the dance; and a dance mode for deciding the dance ofthe player.

FIGS. 21 to 23 are views showing examples of the screens displayed inthe display 2730 while playing the game.

FIG. 21 is a view showing one example of the screen in the memorizingmode, and a mode display 2916 is displayed on the right-hand upperportion of the screen for indicating the memorizing mode. The modedisplay 2916 for indicating the memorizing mode may be the mode displaydisplayed on the right-hand upper portion of the screen as shown in FIG.14 according to the first embodiment.

In the screen of the memorizing mode, the movements of an instructorcharacter 2956 show the movements indicating the correct dance. As amatter of fact, the movements of the instructor character are reversedsideways, but the description will be made assuming that the movementsof the instructor character are correct, for simplicity of description.While the screen of this memorizing mode is displayed, the playermemorizes the correct dance.

In the screen of the memorizing mode and the dance mode, there arecommonly displayed a score 2910, a level gauge 2914, a timing gauge 2902and a window 2958.

The score 2910 indicates the accumulative score till now. The levelgauge 2914 indicates the player's dance level that has accumulated thedecision results of the dance (movements) of the player, and this dancelevel rises when the player's movements are decided to be the correctmovements, and falls when the player's movements are decided to be theincorrect movements. The timing gauge 2902 is constructed such that theneedle is turned at a speed according to the tempo of the dance, andplays the role of a kind of metronome. The window 2958 displays theimage imitating the movement result of the player, detected by the imagesensor 2720.

FIG. 22 is a view showing one example of the screen of the dance mode,and the mode display 2916 indicates that it is the dance mode.

In the screen of the dance mode, there is displayed the correct dance bythe instructor character 2956, and the movement of the player isdetected by the image sensor 2720 so that it is decided whether or notthe player is performing the correct dance.

FIG. 23 is a view showing one example of the screen in which thedecision result in the dance mode is displayed. As shown in FIG. 23, the“Excellent” for deciding the correct motions is displayed in a decisionresult display 2912 on the right-hand of the screen.

In the dance mode, whether or not the dance (movements) of the player iscorrect is decided at every given time interval, at every movement ofthe assigned dance, at every tempo of the BGM and so on, and thedecision result is displayed. As this decision result, there aredisplayed not only the “Excellent” but also the “Good” for indicatingthat the player's movement is generally correct, the “Poor” forindicating that the player's movement is not much correct, and the“Failure” for indicating that the player's movement is incorrect, likethe case according to the first embodiment, as described above.

According to the game apparatus 2700, the dance game is progressed byexecuting the memorizing mode and the dance mode repeatedly. For theplayer 1800, the free movements are reflected on the score or the like,so that the player 1800 can enjoy the dance game physically with theentire body without any restriction on the inputting actions.

Here will be described the principle for detecting and deciding themovements of the player. One example of the functional block diagram ofthe game apparatus 2700 of the second embodiment is shown in FIG. 24.The functional block comprises an input operating unit 2100, an imagesensor 2200, a processing unit 2000, a display unit 2300, a speaker 2400and data storage medium 2500.

The input operating unit 2100 comprises buttons for instructing andinputting the start of the dance game, the level of difficulty of theassigned dance and so on, like the input operating unit 1100 of the gameapparatus 1700 according to the first embodiment. The operating buttons2710 of the game apparatus 2700 shown in FIG. 1 corresponds to the inputoperating unit 2100. The instruction signals inputted by the inputoperating unit 2100 are outputted to the processing unit 2000.

The image sensor 2200 comprises a photo-detection unit 2220, aphoto-detection control unit 2220 and an image analyzing unit 2260, andis realized by an artificial retina chip or the like, like the movementdetection unit 1200 of the game apparatus 1700 according to the firstembodiment. The image sensor 2720 of the game apparatus 2700 shown inFIG. 1 corresponds to the image sensor 2200.

Since the movement detecting technique by this image sensor 2200 hasbeen developed in an earlier development, its detailed description willbe omitted, but its summary will be explained so as to clarify thedifference from the gist of the present invention.

The image sensor 2200 performs an image detection process for detectingthe image, and a characteristic extraction process for extracting thecharacteristics of the image detected, wherein the extraction process isexemplified by: an edge detecting process for detecting the contour ofan object in the image; a resolution varying process by reading pixelsaltogether; a pattern matching process for extracting the object fromthe image; and a random accessing process for detecting only anarbitrary portion in the image. Further, the image sensor 2200 detectsthe movements of the player on the basis of the processed image, but thedescription of the detailed processes by the image sensor 2200 will beomitted because they are like various processes by the photo-detectionunit 1220, the photo-detection control unit 1240 and the image analyzingunit 1260 of the movement detection unit 1200 on the game apparatus 1700according to the first embodiment.

Further, the image sensor 2200 detects the movements of the object. Themovements thus detected by the image sensor 2200 will be described as amovement vector composed of a starting point, a direction, a magnitudeand a speed.

Specifically, the image sensor 2200 monitors and detects the movementvector of the player at all times, and outputs the detected movementvectors temporarily to the processing unit 2000. On the other hand, theimage sensor 2200 detects not only the movement vector but also theimage itself of the player, and outputs the image of the playeradditionally to the processing unit 2000.

The processing unit 2000 comprises mainly a game operation unit 2010, animage generation unit 2030 and a sound generation unit 2040.

The game operation unit 2010 executes the dance game by reading a gameprogram 2510 from the data storage medium 2500, and performs a processfor causing the instructor character to move (or dance) in an objectspace on the basis of a character movement data 534 and a processrelating to a game progress or a score calculation such as a similaritydecision or the like to compare the movement vectors inputted from theimage sensor 2200 and a decision reference data 2536.

The game operation unit 2010 comprises an assigned dance setting unit2014, a movement specifying unit 2017 and a similarity decision unit2016.

The assigned dance setting unit 2014 performs a process for determiningan assigned dance, a music for the BGM and a tempo from the assignmentdata 2530, in accordance with the level of difficulty selected andinputted from the input operating unit 2100.

The specific process are: to determine those corresponding to theassigned dance from the character movement data 2534 for causing theinstructor character to play the correct dance and the decisionreference data 2536 for deciding the dance of the player; to determinethe music for the BGM of the considered assigned dance from the sounddata 2532; and finally to determine the tempo for arranging the speed ofthe assigned dance and the speed of the music. The following descriptionwill made for simplicity by assuming that the assigned dance has twokinds of a dance 1 and a dance 2.

FIG. 25 is a diagram showing one example of the character movement data2534. In FIG. 25, the character movement data 2534 stores the positiondata in the object space, as corresponding to the individual parts ofthe instructor character every key frame, in the dance 1 and the dance2.

For example, the position data of the frame 1 corresponding to a part a(e.g., a right elbow) of the instructor character in the dance 1 have anX-coordinate of Xa1, a Y-coordinate of Ya1 and a Z-coordinate of Za1. Onthe other hand, the position data of the frame 2 have an X-coordinate ofXa2, a Y-coordinate of Ya2 and a Z-coordinate of Za2. Here, themovements of the instructor character between the individual frames areinterpolated.

FIG. 26 is a diagram showing one example of the decision reference data2536. In FIG. 26, the decision reference data 2536 stores the decisionreference data of the direction, the magnitude and the speedcorresponding to the individual regions in the dance 1 and the dance 2at every given timings (or times). Here, the regions are the fourregions divided into four from the range which can be detected by theimage sensor 2200.

The range to be detected by the image sensor 2200 is specified in FIG.27. In this range: the left-hand upper region will be called the firstregion; the right-hand upper region will be called the second region;the left-hand lower region will be called the third region; and theright-hand lower region will be called the fourth region. FIG. 28 showsthe partitions of the individual regions by single-dotted lines. Here,FIGS. 27 and 28 show the contour of the player conveniently for definingthe detection ranges of the image sensor 2200. As a matter of fact,however, what is detected by the image sensor 2200 is the movementvectors. Where the player has no action in FIGS. 27 and 28, therefore,the movement vectors are not detected.

In FIG. 26, the decision reference data of the first region in the dance1 at the time t1 have a direction α11, a magnitude β11 and a speed γ11.On the other hand, the decision reference data at the time t2 have adirection α12, a magnitude β12 and a speed γ12. Thus, the decisionreference data are stored for every regions, because the degree ofsimilarity to the dance (movement) of the player is decided for everyregion, as will be described hereinafter.

The given timings (times) such as the time t1 and the time t2 are thetimings at which the dance of the player is decided, and the movementsof the player are decided on the basis of the decision reference data atthose timings and at every regions. Therefore, those given timings(times) can be adjusted to the tempos of the assigned dance or themusic, determined by the assigned dance setting unit 2014, and can bearbitrarily determined for every given time intervals. If these giventimings are adjusted to the tempos of the assigned dance or the music,it is possible to decide whether or not the movements of the playeraccord to the tempos.

The movement specifying unit 2017 performs the process for specifyingthe movement vectors corresponding to the individual regions, from themovement vectors temporarily inputted from the image sensor 2200. Thedata of the movement vectors, detected for every region, will be calledthe “player's movement data”.

FIG. 29 is a diagram showing one example of the player's movement data.

When the player performs a plurality of movements, as exemplified byFIG. 30A, in which the player raises two arms concurrently, the imagesensor 2200 detects a plurality of movement vectors relating to theelbows and two hands of the arms, as indicated by arrows in FIG. 30B. Inother words, the number of movement vectors inputted from the imagesensor 2200, is dynamically varied according to the movements of theplayer.

The movement specifying unit 2017 specifies what of the first to fourthregions the movement vectors correspond to, from the starting points ofthe movement vectors. When the plurality of movement vectors present inthe same region, the movement specifying unit 2017 specifies onemovement vector in terms of the direction, the magnitude and the speedof the movement vector. Here, for the movement vectors across theregions, the movement vectors for each region can be specified by theaforementioned method. However, the movement vectors across the regionsmay be divided for every region so that the divided movement vectors maybe specified as those corresponding to the individual regions.

For example, when the two arms are concurrently extended in the secondregion, as shown in FIG. 31A, the image sensor 2200 outputs, forexample, a plurality of movement vectors relating to the elbows, the twohands of two arms and so on, as indicated by the arrows in FIG. 31B, asthe movement vectors of the two arms.

Thereafter, the movement specifying unit 2017 specifies such one ofthose movement vectors as located in the second region, according togiven conditions, such as the movement vector having the largestmagnitude, the movement vector having moved at the highest speed, or themovement vector having moved the closest to the transverse directions.

The direction, the magnitude and the speed of the movement vectors, asdetected by the movement specifying unit 2017, at the individual regionsare employed as the player's movement data. In other words, theseplayer's movement data are temporarily updated.

According to the movement specifying unit 2017, it has been describedthat one movement vector is specified where a plurality of movementvectors are concurrently present in each region. However, it isarbitrary to take an average among the movement vectors existing in thatregion. In case of the dance in which two or more movements areconcurrently performed in the same region, it is arbitrary to preparethe decision reference data 2536 corresponding to the individualmovements at the considered timing and to set the number of the player'smovement data for the considered region to the number corresponding tothe number of the considered movements. As a result, the danceperforming two or more actions concurrently in the same region can bedecided more precisely.

The similarity decision unit 2016 decides the degree of similaritybetween the decision reference data 2536 corresponding to the assigneddance and the player's movement data. When the assigned dance is thedance 1 and it is the time t1, for example, the similarity between thedecision reference data 2536 of the dance 1 at the time t1 and theplayer's movement data at the considered point of time is decided forevery regions.

The specific method of deciding the similarity can be exemplified by thepattern matching method of considering the direction, the magnitude andthe speed synthetically or the method of deciding the similarity basedon only the element of the direction for the dance of beginners but ofdeciding the similarly based on all the elements of the direction, themagnitude and the speed for the dance of the senior players. Thus, theinvention can be applied to any of those methods.

In the similarity decision, on the other hand, the timing may be adoptedas one element. More specifically, where the movements are not timed inthe least, a strict decision may be made (by 0 point, for example).Within an allowable time, the score of the decision result may behalved. If well timed, the score of the decision result may be doubled.

For example, by this timing is adjusted to the tempo of the music, it ispossible to be decided whether or not the movement of the player accordsto the tempo.

FIG. 32 shows an example of the timed case, and FIG. 33 shows an exampleof the case in which the movement is not timed but detected within theallowable time. In both FIGS. 32 and 33: the lapse of time is indicatedin the rightward direction; the thick portions on the time axis indicatethe time band of the allowable time; and the times t1, t2 and t3indicate the timings.

In FIGS. 32 and 33, the player performs the movements to lower theextended arm, then to bend the elbow of the lowered arm, and further toextend the bent arm. The image sensor 2200 detects the movement vectors(arrows), as corresponding to the individual movements. At this time, inFIG. 32, there are performed the movements which are timed to the timest1, t2 and t3. FIG. 33 shows that the movements are performed out of thetimings but within the allowable times t1, t2 and t3.

The realizing method using the timings as one element for deciding thedegree of similarity is based on the following principle.

The similarity decision unit 2016 holds the decision reference data 2536at the given timing (at the time t2, for example) considered, beforethat the timing (the time t2) comes. When the time then reaches withinthe allowable time period of the given timing (the time t2) considered,the similarity between the decision reference data 2536 and the player'smovement data is started. Moreover, the decision results are temporarilystored with the time in the memory or the like. However, where adecision result having a higher similarity than the stored one, thestored decision result is updated. This processing is continued till theallowable time elapses. After lapse of the allowable time, thesimilarity decision unit 2016 decides whether or not the time of thestored decision result coincides with the given timing (the time t2),and the decision result is changed according to the coincidence with thegiven timing. In other words, the coincidence weighs the similarity ofthe decision result. The invention is realized by the method thus fardescribed.

The image generation unit 2030 performs the processes for generating animage of the game execution screen, as has been described with referenceto FIGS. 21 to 23.

The process by the image generation unit 2030 are: to generate the imageas viewed from a given view point, of the instructor character dancingin the object space set by the game operation unit 2010; to generate theimage for displaying the score 2910 and the timing gauge 2902; and togenerate the window 2958 for displaying the movement result in animitation. The images generated are displayed in the display unit 2300.

The sound generation unit 2040 performs the processes for generatingmusic on the basis of the sound data 2532 determined by the assigneddance setting unit 2014, at the tempo determined by the assigned dancesetting unit 2014, and for outputting the music to the speaker 2400.Here, the sound data 2532 has been described as the music for the BGM ofthe assigned dance but may be exemplified by voices for the rhythm. Inthis case, the sound generation unit 2040 generates the voices accordingto the tempo determined by the assigned dance setting unit 2014.

The aforementioned functions of the processing unit 2000 can be realizedby the hardware such as the CPU of the CISC type or the RISC type, theDSP, or an IC dedicated to the image fetching, like the processing unit100 of the game apparatus 1700 according to the first embodiment.

The display unit 2300 displays the image an so on, generated by theimage generation unit 2030, and is realized by, for example, the CRT,the LCD, the plasma display or the like. The display 2730 in the gameapparatus 2700 of FIG. 1 corresponds to the display unit 2300.

The data storage medium 2500 stores not only the game program 2510relating to the dance game but also the assignment data 2530 containingthe sound data 2532, the character movement data 2534 and the decisionreference data 2536, and so on. The functions of the data storage medium2500 can be realized by the hardware such as the CD-ROM, the gamecassette, the IC card, the MO, the FD, the DVD, the memory or the harddisk, like the data storage medium 1500 of the game apparatus 1700according to the first embodiment.

As described above, the processing unit 2000 performs the variousoperations on the basis of the program, data, and so on, stored in thedata storage medium 2500.

Here will be described the processes of the game apparatus 2700 of thesecond embodiment. FIGS. 34A and 34B are flow charts showing one exampleof the process in the game apparatus 2700 of the second embodiment.

In FIGS. 34A and 34B, when the game level of difficulty and so on isinputted from the input operating unit 2100, the assigned dance settingunit 2014 sets an assigned dance according to the game level ofdifficulty (step S201).

At this time, the assigned dance setting unit 2014 determines the sounddata 2532, the character movement data 2534, the decision reference data2536, corresponding to that question dance and the tempo of the assigneddance and the music for the BGM.

Next, by the game operation unit 2010 and the image generation unit2030, there are performed the processes on the memorizing mode, i.e.,the processes for displaying the movements of the assigned dance by theinstructor character (step S202).

Thereafter, when the dance mode is started (step S203), the movementspecifying unit 2017 specifies the movement vectors corresponding toeach region from the movement vectors inputted from the image sensor2200 (step S204), to make the player's movement data (step S205).

Next, when the similarity decision unit 2016 decides that the time iswithin the allowable time for deciding the similarity (step S206; YES),the routine transfers to the process of the step S207 in FIG. 34B, asfollowing the arrow 2, and then the similarity decision unit 2016 turnsOFF the similarity decision flag (step S207), and decides the similarityby comparing the player's movement data and the decision reference data2536 (step S208). Here, the similarity decision flag is a flag fordeciding whether or not the similarity is decided, and is stored in thememory or the like.

Thereafter, the similarity decision unit 2016 decides whether or not thesimilarity decided at the step S208 is to be replaced by the decidedsimilarity or updated, so as to determine the highest similarity in theallowable time (step S209). When no updating is decided (step S209; NO),the routine transfers to the process of the step S204 in FIG. 34A, asfollowing the arrow 1, and when the updating is decided (step S209;YES), the similarity is updated (step S210) and the routine transfers tothe process of the step S204 in FIG. 34A, as following the arrow 1.Here, the similarity is stored together with the decided time in thememory or the like.

When the similarity decision unit 2016 decides that the time is notwithin the allowable time (step S206; NO), the similarity decision unit2016 decides whether or not the similarity decision flag is ON (stepS211). When the decision is ON (step S211; YES), the routine transfersto the process of the step S202.

On the other hand, when the decision is not ON (step S211; NO), thesimilarity decision unit 2016 decides the decision time of thesimilarity decided at the step S208 or updated at the step S209 iscoincide with a predetermined timing (step S212). By changing thesimilarity according to the coincidence, moreover, the similarity atthat predetermined timing is decided, and the decision result isdisplayed on the display unit 2300 through the image generation unit2030. Thereafter, the game operation unit 2010 processes the scorerelating to the decided similarity (step S213).

Next, the similarity decision unit 2016 turns ON the similarity decisionflag (step S214), and after the game operation unit 2010 decides whetheror not the assigned dance has ended (step S215). When the decision is NO(step S215; NO), the routine transfers to the process of the step S202.On the other hand, when it is decided that the assigned dance is ended(step S215; YES), the score process to calculate and display the finalpoint is performed (step S216), to end the game.

Next, one example of the hardware configuration capable of realizing thegame apparatus 2700 according to the second embodiment will be describedwith reference to FIG. 35. The game apparatus 2700 shown in FIG. 35, aCPU 2600, a ROM 2602, a RAM 2604, a data storage medium 2606, a soundgeneration IC 2608, an image generation IC 2610 and I/O ports 2612, 2614and 2628, wherein they are connected with one another by a system bus2616 so that the data can be mutually inputted and outputtedtherebetween. Moreover: a display 2618 is connected with the imagegeneration IC 2610; a speaker 2620 is connected with the soundgeneration IC 2608; a control device 2622 is connected with the I/O port2612; a communication 2624 is connected with the I/O port 2614; and animage sensor 2628 is connected with the I/O port 2628.

The data storage medium 2606 stores mainly the program, the image datafor expressing a display, the sound data, the play data and so on, andcorresponds to the data storage medium 2500 shown in FIG. 24. Forexample, like the data storage medium 1606 of the game apparatus 1700according to the first embodiment, when the second embodiment isrealized as the computer, the data storage medium for storing the gameprogram and so on is exemplified by the CD-ROM, the DVD, or the like.When as the consumer game machine, additionally, the game cassette orthe like, is used as the data storage medium. Further, as the arcadegame machine, the memory such as the ROM or the hard disk is used, andin the case, the data storage medium 2606 is the ROM 2602.

The control device 2622 corresponds to the game controller, the controlpanel or the like, for the player to input the instructions of thedifficulty, the play start of the game and so on, to the game apparatus,like the control device 1622 of the game apparatus 1700 according to thefirst embodiment.

The CPU 2600 controls the entire apparatus and processes the variousdata, according to the program stored in the data storage medium 2606,the system program (e.g., the initialization information of the gameapparatus) stored in the ROM 2602, the signals inputted from the controldevice 2622, and so on, like the CPU 1600 of the game apparatus 1700according to the first embodiment.

The RAM 2604 is storage means to be used as the working area of the CPU2600 and is stored with the given contents of the data storage medium2606 and the ROM 2602, the processed results of the CPU 2600, and so on,like the RAM 1604 of the game apparatus 1700 according to the firstembodiment.

The game apparatus further comprises the sound generation IC 2608 andthe image generation IC 2610 so that it can output the proper game soundand image, like the sound generation IC 1608 and the image generation IC1610 of the game apparatus 1700 according to the first embodiment.

The sound generation IC 2608 is an integrated circuit for generating themusic of the BGM or the like, the voices for the rhythms of the dance,and so on, on the basis of the data stored in the data storage medium2606 or the ROM 2602, and the generated music or the like is outputtedby the speaker 2620.

The image generation IC 2610 is an integrated circuit for generating thepixel data to be outputted to the display 2618, on the basis of theimage data sent from the RAM 2604, the ROM 2602, the data storage medium2606 or the like.

The image sensor 2628 is constructed of the artificial retina chip orthe like and detects the movements of the player, the image of theplayer, and so on.

The display 2618 and the communication device 2624 are constructed andhave functions like the display 1618 and the communication device 1624of the game apparatus 1700 according to the first embodiment,respectively. Therefore, their descriptions will be omitted.

The various processes as having been described with reference to FIGS.22 to 33, are realized by the data storage medium 2606 which stores theprogram for performing the processes and so on, shown in the flow chartof FIG. 34, and by the CPU 2600, the image generation IC 2610 and thesound generation IC 2608 which act according to said program, and so on.

Here, the processes to be performed in the image generation IC 2610 andso on may be performed in the software manner by the CPU 2600 or ageneral-purpose DSP or the like.

Now, the game apparatus 2700 according to the second embodiment can beapplied to the arcade game machine as shown in FIG. 19A and FIG. 19B,like the case of the game apparatus 1700 according to the firstembodiment. Herein, the description of the case wherein the gameapparatus 2700 is applied to the arcade game machine will be omitted.

As described hereinbefore, according to the game apparatus 2700according to the second embodiment of the present invention, the freemovements of the player are reflected as they are on the score or thelike. Accordingly, it is possible to provide a physical game apparatuswherein the player uses the entire body and has no restriction on theinputting actions.

Here, the invention should not be limited to the second embodiment thusfar described but can be modified in various manners.

For example, the foregoing second embodiment has been described byexemplifying the dance game. Since it is one gist of the presentinvention how the movements of the player are detected and how thesimilarity of the assigned movements is decided. Accordingly, thepresent invention can be applied to such a game as, for example, agesture game, wherein the game is required to decide the movements ofthe player.

Further, the description has been made assuming that the range to bedetected by the image sensor 2200 is partitioned into the four regions.However, the regions may be either four regions or more so that thesimilarity for a finer movement may be decided, or less than fourregions. And further, the number of regions may be so changed midway ofthe assigned dance that the region is one in an intermission of themusic. Furthermore, the number, the size and the position of the regionsmay be changed according to the kind of the assigned dance, or thestanding position or the lengths of the limbs of the player, detected bythe image sensor.

Furthermore, the description of specifying the movement vectors everyregion, has been made. However, the four movement vectors may bespecified in the order of the larger movements or the higher speeds ofthe movements of all the regions.

Furthermore, the movements of the player have been shown and describedonly on the movements of arms. However, the movements should not belimited to those of the two arms or feet but may be exemplified by thesideway movements, squats or jumps of the player. No matter what themovements of the player might be, the similarity of the movements isdecided on the basis of the movement vectors relating to those movementsso that the movements of the player are not limited.

Furthermore, it has been explained that the decision of the similarityis described, first, by determining the highest decision result ofsimilarity within the allowable time and next, by weighing it accordingto the coincidence with the given timing, but vice versa. Specifically,first, the coincidence with the given timing may be decided within theallowable time and next, the similarity of the movement vectors havingthe high coincidence may be decided.

Furthermore, according to the body shape such as the height or thelengths of the limbs of the player, it is arbitrary to change thedecision result of the similarity. In a method for realizing this,first, an announcement is made before the start of the dance game to askthe player to take a pose in which the two arms and the two feet areextended, and the game operation unit 2010 detects the height and thelengths of the limbs of the player on the basis of the image, whereinthe player extends the limbs, as detected by the image sensor 2200.Specifically, the length of the arms and the height of the player aredecided from the range of the image, as occupied by the player, suchthat the arm length is decided from the sideway width of that rangewhereas the height is decided from the height of the range. Next, thesimilarity decision result of the dance (movements) in the dance mode isweighed in accordance with the height or the length of the limbs of theplayer. Accordingly, the method can be realized, as described above.

Furthermore, the similarity decision by the similarity decision unit2016 is performed for each region, so that the decision results of theindividual regions may be displayed.

For example, as in one example of the screen shown in FIG. 36, thedisplay color of the considered region of the window 2958 may be changedaccording to the decision result of each region, or the consideredregion may display such a mark as “∘” and “X”. In this case, the playercan instantly recognize where the movement of the region of the dancejust played is bad or good.

Effects according to the second embodiment of the present invention willbe explained, as follows.

According to the present invention according to the second embodiment,the free movements of the player are reflected as they are on the scoreor the like. Accordingly, It is possible to provide a physical gameapparatus wherein the player uses the entire body and has no restrictionon the inputting actions.

Further, the similarity is decided by using the direction, the magnitudeand the speed of the movements of the player as the elements.Accordingly, it is possible to objectively make the subjective decisionon whether or not the movements of the player are good.

Furthermore, the similarity is decided at the given timing or for everydetection regions, so that the processing can be lightened.

Third Embodiment

The game apparatus 3700 which can realize a body sensing game requiringa player to use an entire body and vary contents of the game byoutputting hints, will be explained according to the third embodiment ofthe game apparatus of the present invention, with reference to FIGS. 1,2 and 37 to 53.

A front view showing an external of a housing 3750 of the game apparatus3700 as an arcade game machine according to the third embodiment of thegame apparatus of the present invention, and a vertical sectional viewof the game apparatus 3700 showing a positional relation between aplayer 1800 and an image sensor 3720 and a display 3730 which aredisposed in the housing 3750, are the same as the front view and thevertical sectional view of the game apparatus 1700 according to thefirst embodiment, as shown in FIG. 1 and FIG. 2. Therefore, the views isnot omitted to show in figures.

The game apparatus 3700 comprises an artificial retina camera 3720 fordetecting the movements of the player 1800, a display 3730 fordisplaying a game screen, operating buttons 3710 for inputting a mode ofa game, a level selection, an instruction for starting a game, and soon, a light 3790 for illuminations and a speaker 3760 for outputting aBGM for a dance, voices for taking rhythms, and so on.

The game apparatus 3700 detects a movement of the player by theartificial retina camera 3720, and compares this detected movement ofthe player with a movement of a predetermined assigned dance, to decidethe level of similarity. Thereafter, the game apparatus 3700 progressesthe game and processes the score, according to the decision result.

FIG. 37 is a view showing an exemplary screen in a memorizing mode,displayed on the display 3730. This mode selection screen prompts theplayer 1800 to select the mode of the game the player 1800 is about tostart; for example, the player 1800 may select either a normal mode, areal-time mode or a free mode.

Herein, the normal mode represents a mode wherein a sample dance that isthe lateral inversion of an assigned dance is displayed throughmovements of an instructor character 3956 (see FIG. 39) prior to thedance-start instruction, and the player 1800 then starts dancing.

The real-time mode represents a mode wherein the start of the dance isimmediately instructed as the game starts, and the player 1800 dancessimultaneously with the instructor character 3956 by copying themovements demonstrated by the instructor character 3956 displayed on thescreen.

The free mode represent a mode wherein the player 1800 freely dancesindependently from the dance of the instructor character 3956 displayedon the screen.

FIG. 38 is a view showing an exemplary level selection screen displayedon the display 3730. This level selection screen shown in FIG. 38 isdisplayed only when the normal mode is selected in the mode selectionscreen shown in FIG. 37. This level selection screen prompts the playerto select a desired level from different game levels of difficulty; forexample, one from different game levels of difficulty such as a“Beginner”, “Intermediate level” and “Expert”.

Herein, for simplicity the number of the levels is assumed to be thesethree, “Beginner”, “Intermediate Level” and “Expert”, however, thenumber is not limited to three, there may be 4 or more levels, or 2 orless levels.

FIG. 39 is a view illustrating an exemplary screen displayed on thedisplay 3730 prior to the start of the dance (before the dance-startinstruction is given to the player) when the normal mode has beenselected on the mode selection screen shown in FIG. 37. In the screenshown in FIG. 39, the instructor character 3956 for demonstrating thesample dance (the laterally-inverted movements of the assigned dance) isdisplayed.

Further, in the screen shown in FIG. 39, for example, a word “Memorize”may be indicated on the top right corner of the screen, in order toinstruct the player 1800 to memorize this sample dance shown by theinstructor character 3956, that is the assigned dance, like the screenshown in FIG. 14 according to the first embodiment. The indication toinstruct the player 1800 to memorize the sample dance may be like theindication displayed on the top right of the screen shown in FIG. 21according to the second embodiment.

FIG. 40 is a view illustrating an exemplary screen displayed while theplayer 1800 is dancing (while the player is dancing after thedance-start instruction is given). The instructor character 3956 whoperforms the movements (dance) corresponding to the mode selected on themode selection screen shown in FIG. 37 and the level selected on thelevel selection screen shown in FIG. 38, is displayed on the screen, asshown in FIG. 40. Further, the decision result 3912 corresponding tomovement of the player 1800 detected by the artificial retina camera3720 is displayed on the upper right of the screen.

As for the indication of this decision result, for example, when theplayer's movement is “very similar” to the assigned dance, then a term“Perfect” would be displayed, when the player's movement is “fairlysimilar”, then a term “Good” would be displayed, when the player'smovement is “little similar”, then a term “Poor” would be displayed, andwhen the player's movement is “not similar at all”, then a term“Failure” would be indicated, like the cases according to the first andthe second embodiments.

Further, on the screen during the game performance (the screens shown inFIGS. 39 and 49), a level gauge 3914, a timing gauge 3902 and a score3910 are displayed.

Herein, the level gauge 3914, the timing gauge 3902, and the score 3910have functions like the level gauge 2914, the timing gauge 2901, and thescore 2910 of the game apparatus 2700 according to the secondembodiment, respectively. Therefore, the descriptions of them will beomitted.

The game apparatus 3700 can assign an assigned dance requiring the useof an entire body, and reflect the movements of the entire body of theplayer 1800 movements to the game. Accordingly, the player can enjoyplaying the game by using the entire body to experience a physicalamusement.

FIGS. 41A and 41B are views illustrating the relationship between themovement of the instructor character 3956 displayed on the display 3730and the movement of the player 1800.

FIG. 41A shows the movement of the instructor character 3956 displayedon the display 3730, and FIG. 41B shows the player 1800 copying themovement of the instructor character 3956 shown in FIG. 41A.

The instructor character 3956 displayed on the display 3730 performs amovement that is a lateral inversion of a movement actually assigned tothe player 1800. Accordingly, for example, when the instructor character3956 raises its left arm as shown in FIG. 41A, then the movement of theplayer 1800 raising the right arm as shown in FIG. 41B would be deemedas equivalent to the movement demonstrated by the instructor character3956.

That is, for example, in a case wherein the player 1800 memorizes thesample dance demonstrated by the instructor character 3956, orsimultaneously dances with the instructor character 3956 by copying thesample dance, the player 1800 does not have to try to determine whetherthe instructor character 3956 has been the right arm or the left arm,and the player 1800 only has to perceive the moved arm from theviewpoint of the player 1800. Accordingly, the player 1800 may movereflectively in response to the movement of the instructor character3956 as viewed by the player.

FIG. 42 is an exemplary functional block diagram of the game apparatus3700 according to the third embodiment of the present embodiment. Thefunctional block of the third embodiment comprises an input operatingunit 3100, an image sensor 3200, a display unit 3300, a speaker 3400, aprocessing unit 3000 and a data storage medium 3500.

The input operating unit 3100 has buttons for instructing and inputtingthe selection of the mode, the selection of the level, the start of thegame, and so on, for the dance game, like the input operating units 1100and 2100 of the game apparatuses 1700 and 2700 according to the firstand second embodiments. The operation buttons 3710 of the game apparatus3700 shown in FIG. 1 corresponds to the input operating unit 3100. Theinstruction signals inputted by the input operating unit 3100 isoutputted to the processing unit 3000.

The image sensor 3200 detects a movement of the player 1800 as amovement vector having the components of a starting position, adirection, a magnitude and a speed by performing the processes like theprocesses by the movement detection unit 1200 and the image sensor 2200of the game apparatuses 1700 and 2700 according to the first and secondembodiments. Thereafter, the image sensor 3200 outputs the movementvector detected to the processing unit 3000. Further, the movementdetection function provided by the image sensor 3200 may be realized byan artificial retina chip or the like.

The image sensor 3200 detects the movement of the player 1800, andthereby the operation time required for the feature extractionprocessing by the processing unit 3000 may be reduced, so that it ispossible to accelerate the process of the apparatus as a whole. That is,in an apparatus according to an earlier development, the movement of theplayer is detected based on an image captured by the imaging means suchas a CCD or the like, so that a processing speed that is practical for agame apparatus has not been achieved. The solution to this problem isrealized through the present invention.

The processing unit 3000 comprises a game operation unit 3010, an imagegeneration unit 3030 and a sound generation unit 3040.

The game operation unit 3010 performs: a process for setting the gamemode and the level based on the operation data inputted from the inputoperating unit 3100; a process for proceeding the game based on a gameprogram 3510; a process for instructing the start of the dance through avisual display or a voice etc.; a process for setting an assigned danceto be assigned from an assignment data 3530; a process for moving(dancing) the instructor character 3956 in an object space based on acharacter movement data 3534; a process for controlling the position anddirection of the viewpoint (the virtual camera 3542) based on a virtualcamera control data 3540; a process for deciding the similarity betweenthe movement of the player and the assigned dance by comparing themovement vector inputted from the image sensor 3200 and a decisionreference data 3536; and a process for calculating the score based onthe decision result.

Further, when the game operation unit 3010 performs the process forsetting the assigned dance, it first performs a process to determine theassigned dance, the music for the BGM and the tempo from the assignmentdata 3530. More specifically, it determines those corresponding to theassigned dance from the character movement data 3534 and the decisionreference data 3536, and at the same time, determines a music to serveas a BGM of the assigned dance from the sound data 3532, and it finallyperforms the process to determine the tempo to match the speed of theassigned dance with the speed of the music.

The game operation unit 3010 comprises a similarity decision unit 3016,a character movement setting unit 3012 and a viewpoint control unit3019.

The character movement setting unit 3012 sets a character controlpattern for controlling the movements of the instructor character 3956and a camera control pattern for controlling the camera position etc. ofthe virtual camera 3542 (a given viewpoint (see FIGS. 45A and 45B) at atime of image generation of the instructor character 3956 by the imagegeneration unit 3030) on the basis at the control data inputted from theinput operating unit 3100 and the control pattern data 3538.

Further, the character movement setting unit 3012 also controls theinstructor character 3956 to move within the object space in accordancewith the set character control pattern and the character movement data3534.

FIG. 43 is a diagram showing an exemplary control pattern data 3538. Asshown in FIG. 43, the control pattern data 3538 contains a charactercontrol pattern and the camera control pattern according to the mode andlevel for the state prior to the start of the dance (the state beforethe dance-start instruction is given to the player) and the state duringthe dance (the state after the dance-start instruction is given to theplayer, or the state during which the player performs the dance).

For example, where the normal mode and the expert's level have beenselected, the character control pattern and the camera control patternprior to the dance performance are “A” and “1” respectively, and thoseduring the dance performance are “B” and “1” respectively.

FIG. 44 is a diagram showing an exemplary character movement data 3534.As shown in FIG. 44, the character movement data 3534 contains positiondata indicated by the coordinates data, of the respective body parts ofthe instructor character 3956 (for example, the body parts correspondingto the right wrist, left knee . . . etc.) within the object space foreach respective key frame, representing the movements of the instructorcharacter 3956 corresponding to the character control patterns “A” and“B” associated with the selected mode or level indicated in the controlpattern data 3538 shown in FIG. 43.

For example, the position data for the frame 1, which corresponds to thebody part “a” of the instructor character 3956 for the character controlpattern “A” is the coordinates (Xa11, Ya11, Za11), and the position datafor the frame 1, which corresponds to the body part “a” of theinstructor character 3956 for the character control pattern “B” is thecoordinates (Xa21, Ya21, Za21). The movements of the instructorcharacter 3956 between the respective frames are interpolated by thecharacter movement setting unit 3012.

As shown in FIG. 44, the character movement data 3534 is set so that thepositional coordinate data for the respective body parts for thecharacter control pattern “A” would allow the instructor character 3956to perform the movements (the sample dance) that are lateral inversionof the assigned dance, and the positional coordinate data for therespective body parts for the character control pattern “B” would allowthe instructor character 3956 to perform the movements having noassociation with the assigned dance.

That is, where the normal mode is selected, the character movementsetting unit 3012 controls the instructor character 3956 to perform thesample dance prior to the dance performance. Further, where thebeginner's level or the intermediate level is selected in the normalmode, or where the real-time mode is selected, the character movementsetting unit 3012 controls the instructor character 3956 to demonstratethe sample dance during the dance performance. And further, where theexpert's level is selected in the normal mode, or where the free mode isselected, the character movement setting unit 3012 controls theinstructor character 3956 to perform the movements not associated withthe assigned dance during the dance performance.

It is of course possible to provide a greater number of charactercontrol patterns in the character movement data 3534 to assign a greaternumber of movements to the instructor character 3956 instead of only “A”and “B”.

The viewpoint control unit 3019 controls the position, the angle and theangle of view of the virtual camera 3542 based on the camera controlpattern set by the character movement setting unit 3012.

FIGS. 45A and 45B are views illustrating the virtual camera 3542 withinthe object space. FIG. 45A is a view showing the relationship betweenthe instructor character 3956 and the virtual camera 3542 within theobject space as set by the character movement setting unit 3012, andFIG. 45B is a view showing the angle of the virtual camera 3542.

As shown in FIG. 45A, the position of the virtual camera 3542 is definedby the distance from the instructor character 3956 and the latitude andthe longitude on a sphere 3544 (FIG. 45A shows only an upper sphere, forsimplicity) which has a center at the instructor character 3956 and aradius as the distance from the instructor character 3956. Herein, thelatitude is represented by an angle defined by the center of the sphere3544 and the virtual camera 3542 with the horizontal plane being 0degree. The longitude is represented by an angle defined by a planecontaining a given reference meridian of the sphere 3544 and a planecontaining a meridian passing through the virtual camera 3542.

As shown in FIG. 45B, the angle of the virtual camera 3542 is defined byan angle Vx representing a vertical tilt, an angle Vy representing alateral tilt and an angle Vz representing a rotation angle about theviewing direction.

FIG. 46 is a diagram showing a data configuration of the virtual cameracontrol data 3540.

The virtual camera control data 3540 comprises the data corresponding tothe position, the direction and the angle of view for each giventemporal interval such as each respective key frame, as the control dataof the virtual camera 3542 corresponding to the camera control pattern“1” and “2” that are set by the control pattern data 3538 shown in FIG.43.

In FIG. 46, according to the camera control pattern “1”, the position ofthe virtual camera 3542 within the object space is a position in frontof the instructor character 3956, and the distance from the instructorcharacter 3956, the latitude and the longitude are indicated by Vd11,Vi11 and Vk11 respectively. The direction of the virtual camera 3542 isdefined by the vertical tilt angle Vx11, the lateral tilt angle Vy11 andthe rotation angle Vz11 about the viewing direction. The angle of viewis Z11, and the position, the direction and the angle of view are allfixed values.

On the other hand, in FIG. 46, according to the camera control pattern“2”, the position of the virtual camera 3542 within the object space isso configured to allow the virtual camera 3542 to revolve around theinstructor character 3956. The direction of the virtual camera 3542 isset by fixed values of the vertical tilt angle Vx21, the lateral tiltangle Vy21 and the rotation angle Vz21 about the viewing direction. Theangle of view is set to a fixed value Z21.

That is, prior to the dance performance, when the normal mode has beenselected, the instructor character 3956 would be displayed to be facingforward at a fixed position and in a fixed size.

During the dance performance, when the beginner's level or the expert'slevel has been selected in the normal mode, or the real-time mode hasbeen selected, the instructor character 3956 would be displayed to befacing straight forward at a fixed position and in a fixed size, andwhen the intermediate level has been selected in the normal mode, or thefree mode has been selected, although the size of the instructorcharacter 3956 displayed would be fixed, the instructor character 3956would be turning as displayed.

FIGS. 47 to 49 are views illustrating exemplary displays of theinstructor character 3956 displayed in accordance with the charactermovement pattern and the camera movement pattern corresponding to theselected mode and level.

FIG. 47 is an exemplary display of the instructor character 3956 who isperforming the movement (the sample dance) that is the lateral inversionof the assigned dance, FIG. 48 is an exemplary display of the instructorcharacter 3956 performing the sample dance, in which the instructorcharacter 3956 is turning as displayed, and FIG. 49 is an exemplarydisplay of the instructor character 3956 performing a movement having noassociation with the assigned dance.

When the normal mode has been selected, prior to the dance performance,as shown in FIG. 47, the instructor character 3956 facing forward at afixed position and in a fixed size would be displayed and perform thesample dance.

When the beginner's level has been selected, during the danceperformance, as shown in FIG. 47, the instructor character 3956 facingforward at a fixed position and in a fixed size would be displayed andperform the sample dance, in the same manner as the manner prior to thedance performance. Accordingly, the game level of difficulty would below, so that even a beginner player may enjoy pleasure of playing thegame.

When the intermediate level has been selected, during the danceperformance, as shown in FIG. 48, the instructor character 3956 showingthe sample dance would turn as displayed. Therefore, the displayedinstructor character 3956 would also face the sides and the back.Accordingly, although the player would easily recognize whether theplayer should move a leg or an arm, the player may be misled in judgingthe right or left since the instructor character 3956 is turning. Also,the instructor character 3956 may face one side as it turns as shown inFIG. 48, and in this state, the player only receives information on theone side of the instructor character 3956. Accordingly, the game levelof difficulty would be greater than the beginner's level.

When the expert's level has been selected, as shown in FIG. 49, thedisplayed instructor character 3956 would be making movements having noassociation with the assigned dance. Therefore, the player would receiveno clue that hints at the assigned dance during the dance performance.In addition, the player may mistakenly engage in a movement that isdifferent from the assigned dance by being misled by the movements ofthe instructor character 3956. Accordingly, the game level of difficultywould become further greater than the intermediate level.

In this way, by changing the display of the instructor character 3956during the dance performance, it is possible that the game level ofdifficulty is changed. As a result, it is possible to realizes a gameamusing to beginners as well as skilled players, which will not letthose skilled players get easily bored.

When the real-time mode has been selected, during the dance performance,as shown in FIG. 47, the instructor character 3956 facing forward at afixed position and in a fixed size would be displayed and show thesample dance. However, the player would not be supplied with theinformation on the assigned dance prior to the dance performance, sothat the player must start dancing without any advance knowledge of theassigned dance. Accordingly, the game level of difficulty would behigher, and the player would feel uncertainty and be urged to be tenser.Consequently, the amusement of the game is further enhanced.

When the free mode has been selected, the instructor character 3956would not be displayed before the dance performance, and the instructorcharacter 3956 who is turning and performing movements not associatedwith the assigned dance would be displayed during the dance performanceas shown in FIG. 49. Therefore, in the state wherein the information onthe assigned dance would not be presented to the player before andduring the dance performance, the player is required to perform a freedance. Thereby, for this player's free dance, the score, the decisionresult and the level are indicated. Accordingly, the player may feel asif the player's dance is being decided. Consequently, a type ofamusement that is different from the normal mode or the real-time modein which the player dances in accordance with the assigned dance may beprovided.

In FIG. 46, for example, the camera control pattern “2” may be soconfigured to provide parallel displacement of the position of thevirtual camera 3542 in horizontal and vertical directions in front ofthe instructor character 3956. In this case, the instructor character3956 displayed on the screen would move horizontally or vertically whilefacing forward.

Further, in FIG. 46, for example, the direction and the angle of view ofthe virtual camera 3542 for the camera control pattern “2” may beconfigured to be set variably, instead of using the fixed values. Insuch a case, the size, the position and so on of the instructorcharacter 3956 displayed on the screen would also change.

Furthermore, in FIG. 46, a greater number of camera control patterns maybe provided instead of using only the camera control patterns “1” and“2”. In this case, the game level of difficulty may be changed in agreater variety by combining them with the character control patterns,allowing further divided level setting.

The similarity decision unit 3016 decides the level of similarity bycomparing the respective components of the movement vector inputted fromthe image sensor 3200 with the decision reference data 3536corresponding to the assigned dance.

FIG. 50 is a diagram showing an exemplary decision reference data 3536.The decision reference data 3536 contains a decision reference data ofthe direction, the magnitude and the speed at each given timing (timeperiod). In FIG. 50, the decision reference data at the time t1 for adance 1 provides a direction α11, a magnitude β11 and a speed γ11.

A specific similarity decision method by the similarity decision unit3016 is like the similarity decision method by the similarity decisionunit 2016 of the game apparatus 2700 according to the second embodiment,so that its description will be omitted.

The image generation unit 3030 performs a process for generating imagesof the instructor character 3956 making movements within the objectspace defined by the character movement setting unit 3012, as viewedfrom the virtual camera 3542 (a given viewpoint) that is controlled forits position, its angle and its angle of view by the viewpoint controlunit 3019.

In other words, when the virtual camera 3542 is controlled in accordancewith the set values of the camera control pattern “1” in the virtualcamera control data 3540 by the viewpoint control unit 3019, the imagesof the instructor character 3956 facing forward are generated by theimage generation unit 3030. On the other hand, when the virtual camera3542 is controlled in accordance with the set values of the cameracontrol pattern “2”, the images of the turning instructor character 3956are generated by the image generation unit 3030.

Further, the image generation unit 3030 performs a process forgenerating images of the score 3910, the level gauge 3914 and so on,based on the score calculated by the game operation unit 3010, thesimilarity decision result and so on. The images provided by the imagegeneration unit 3030 are displayed on the display unit 3300.

The sound generation unit 3040 performs a process for generating andoutputting a music from a speaker 3400 based on the sound data 3532selected by the game operation unit 3010, at a tempo determined by thegame operation unit 3010. The sound data 3532 has been explained asbeing a back ground music of the assigned dance, however, it mayalternatively be voices or effect sounds for helping the player to takethe rhythms. In the latter case, the sound generation unit 3040 would begenerating voices or effect sounds that synchronize with the tempodetermined by the game operation unit 3010.

The above-described functions of the processing unit 3000 may berealized by hardware such as a CPU of a CISC-type or a RISC-type, a DSP,an image capturing IC and a memory etc, like the cases of the processingunits 1000 and 2000 of the game apparatuses 1700 and 2700 according tothe first and second embodiments.

The display unit 3300 is a display for indicating images and so ongenerated by the image generation unit 3030, and the display 3730 in thegame apparatus 3700 shown in FIG. 1 corresponds to this display unit3300. Further, the display unit 3300 can be realized by a CRT, a LCD, aplasma display or the like, like the cases of the display units 1300 and2300 of the game apparatuses 1700 and 2700 according to the first andsecond embodiments.

The data storage medium 3500 stores, besides the game program 3510, theabove-explained sound data 3532, the character movement data 3534, thedecision reference data 3536, the assignment data 3530 including thecontrol pattern data 3538, and the virtual camera control data 3540. Thefunction of this data storage medium 3500 may be realized by hardwareincluding a CD-ROM, a game cassette, an IC card, a MO, a FD, a DVD, ahard disk, a memory and so on, like the cases of the data storagemediums 1500 and 2500 of the game apparatuses 1700 and 2700 according tothe first and second embodiments.

As previously explained, the processing unit 3000 performs variousprocesses based on the program, the data and so on stored on this datastorage medium 3500.

The process of the game apparatus 3700 according to the third embodimentof the present invention will now be explained with reference to theflow chart shown in FIG. 51.

With reference to FIG. 51, when a mode is selected and set based on theoperation data inputted from the input operating unit 3100 in the modeselection screen (FIG. 37) (step S301), the game operation unit 3010decides whether or not the set mode is the normal mode (step S302).

When the set mode is detected to be the normal mode at the step S301(step S301; YES), the image generation unit 3030 instructs the displayunit 3300 to display the level selection screen (FIG. 38) thereon, and alevel is selected and set based on the operation data inputted from theinput operating unit 3100 (step S303).

Next, the character movement setting unit 3012 controls the instructorcharacter 3956 to perform movements (sample dance) which is the lateralinversion of the assigned dance, based on the control pattern data 3538and the character movement data 3534, and at the same time, controls thevirtual camera 902 based on the control pattern data 3538 and thevirtual camera control data 3540. The image generation unit 3030generates images of the instructor character 3956 as viewed from thevirtual camera 3542 and displays the images on the display unit 3300(step S304). Upon completion of the sample dance, the game operationunit 3010 instructs the start of the dance performance (step S305).

On the other hand, when the normal mode has not been selected at thestep S302 (step S302; No), the game operation unit 3010 directlyinstructs the start of the dance performance (step S305).

After the dance-start instruction, the character movement setting unit3012 sets the character control pattern and the camera control patternassociated to the set mode and level, that are to be used during thedance performance, based on the control pattern data 3538, and controlsthe instructor character 3956 to perform the movements within the objectspace in accordance with the character movement data 3534 correspondingto the set character control pattern, and also controls the virtualcamera 3542 for its position, its direction, and its angle of viewaccording to the virtual camera control data 3540 corresponding to theset camera control pattern.

Thereafter, the image generation unit 3030 generates the images of theinstructor character 3956 as viewed from the virtual camera 3542, anddisplays the images on the display unit 3300 (step S306).

Next, during the dance performance by the player, the image sensor 3200detects the movements of the player (step S307), and the similaritydecision unit 3016 compares the detected movements of the player and thedecision reference data 3536 to decide the level of similarity (stepS308).

Thereafter, the game operation unit 3010 processes to calculate thescore based on the decision result decided by the similarity decisionunit 3016, and the decision result and the score are displayed on thescreen (step S309). Thereby, the game is terminated.

Next, one example of the hardware configuration capable of realizing thegame apparatus 3700 according to the third embodiment will be describedwith reference to FIG. 51. The game apparatus 3700 shown in FIG. 52,comprises a CPU 3600, a ROM 3602, a RAM 3604, a data storage medium3606, a sound generation IC 3608, an image generation IC 3610, I/O ports3612, 3614 and 3626, wherein they are connected through a system bus3616 so as to be capable of inputting and outputting data to and fromeach other. A display 3618 is connected to the image generation IC 3610,a speaker 3620 is connected to the sound generation IC 3608, a controldevice 3622 is connected to the I/O port 3612, a communication device3624 is connected to the I/O port 1614 and an image sensor 3628 isconnected to the I/O port 3626.

The data storage medium 3606 corresponds to the data storage medium 3500shown in FIG. 42. Further, the data storage medium 3606 is constructedand has functions, like the data storage mediums 1606 and 2606 of thegame apparatuses 1700 and 2700 according to the first and secondembodiments shown in FIGS. 18 and 35.

Further, the control device 3622, the CPU 3600, the RAM 3604, the soundgeneration IC 3608, the image generation IC 3610, the display 3618, andthe communication device 3624, are constructed and have functions likethe control devices 1622 and 2622, the CPU 1600 and 2600, the RAM 1604and 2604, the sound generation IC 1608 and 2608, the image generation IC1610 and 2610, the displays 1618 and 2618, and the communication devices1624 and 2624 of the game apparatuses 1700 and 2700 according to thefirst and second embodiments shown in FIGS. 18 and 35, respectively.Therefore, their descriptions will be omitted.

The image sensor 3628 comprises an artificial retina chip etc., and itdetects the movements, the images and so on of the player, like theimage sensor 2628 of the game apparatus 2700 according to the secondembodiment.

The above various processes explained with reference to FIGS. 37 to 50are realized by the data storage medium 3606 storing a program forperforming the processes indicated in the flow chart shown in FIG. 51and the CPU 3600, the image generation IC 3610 and the sound generationIC 3608 etc. that operate according to the program.

The processes performed by the image generation IC 3610 or the like, maybe written as software applications run by the CPU 3600, a general DSPor the like.

Now, the game apparatus 3700 according to the third embodiment can beapplied to the arcade game machine as shown in FIG. 19A and FIG. 19B,like the cases of the game apparatuses 1700 and 2700 according to thefirst and second embodiment. Herein, the description of the case whereinthe game apparatus 3700 is applied to the arcade game machine will beomitted.

As explained heretofore, according to the game apparatus 3700 of thethird embodiment of the present invention, the movements of the player'sentire body may be reflected to a game, and the game level and the gamecontent may be changed by varying the display of the instructorcharacter 3956 according to the selected level. Accordingly, the contentof the game can be readily changed, and it is possible to realize abody-sensing type game apparatus that allows the player to play by usingthe entire body.

The game apparatus 3700 according to the third embodiment of the presentinvention is not limited to the above embodiment, and variousmodifications are possible.

For example, in the above third embodiment, when the real-time inputmode has been selected the instructor character 3956 has been explainedas being facing forward while it demonstrates the sample dance, however,the direction of the instructor character 3956 is not limited thereto.The instructor character 3956 may be configured to turn, to move, or tochange the size while it is displayed, to increase the game level ofdifficulty for the real-time mode. It may be possible to provide thelevel selection also for the real-time mode, and the display of theinstructor character 3956 may be configured to change according to aselected level.

Further, the image sensor 3200 may be configured to capture also theplayer's image itself. As shown in FIG. 53, a window 3958 may beprovided on the screen displayed during the game performance to displaythe image of the player as captured by the image sensor 3200. In thisway, the player can self-check the own movements (dance).

Furthermore, it may be configured to output indications through thevoice from the speaker 3400 or the textual display in synchronizationwith the movements of the instructor character 3956 to indicate itscurrent movements, before or during the dance performance. In this case,since the movements of the instructor character 3956 are the lateralinversion, for example, when the instructor character 3956 raises itsright arm, a voice or a text display saying “left arm” would beoutputted. Furthermore, by outputting the voices or the textual displayindicating movements that do not match with the assigned dance duringthe dance performance to confuse the player, the game level ofdifficulty would be increased, and thereby the amusement of the game isenhanced.

Furthermore, the game level of difficulty may be changed, for example,by changing the camera control pattern during the dance performancebased on the current value of the level gauge 3914 displayed.

For example, under a condition wherein the intermediate level has beenselected, thus the instructor character 3956 is being displayed with thevirtual camera 3542 revolving around the instructor character 3956 (thecamera control pattern “2”) during the dance performance, if the levelindicated by the level gauge 3914 becomes lower than a given level (forexample, the case wherein the player is tired or the level is too hardfor the player), the virtual camera 3542 may stop at the fixed positionin front of the instructor character 3956.

Conversely, under a condition wherein the beginner's level has beenselected, thus the virtual camera 3542 is in a fixed state at a positionin front of the instructor character 3956, if the level indicated by thelevel gauge 3914 exceeds a given level (for example, the case whereinthe selected level is too easy for the player), the virtual camera 3542starts revolving around the instructor character 3956.

Furthermore, the game level of difficulty has been explained as beingcontrolled by the movements of the instructor character 3956 and themovements of the virtual camera 3542. However, the game level ofdifficulty may be controlled by the speed of the movement of theinstructor character 3956 and the speed (the tempo) of the music. Inthis case, for example, when the camera control pattern “2” is selectedto have the virtual camera 3542 revolve around the instructor character3956, if the speed of the movements of the instructor character 3956 andthe music is faster, the revolving speed of the virtual camera 3542 mayalso be accelerated.

Furthermore, in the third embodiment, it has been explained that theinformation hinting at the assigned dance is not given when the freemode has been selected, however, the present invention is not limited tothese embodiments. For example, the output of the hints for the assigneddance may be configured to stop upon an input of a given instructioninto the input operating unit 3100 and so on even during the game.

Effects according to the third embodiment of the present invention willbe explained, as follows.

According to the present invention, the movements of the player's entirebody may be reflected to the game, and thereby it is possible to realizea body-sensing type game allowing the player to play by using theplayer's entire body.

Further, since the game level of difficulty and the content of the gamecan readily be changed by varying the timing and the contents of thehint indications or the voice outputs, a game enjoyable to any playersfrom beginners to experts can be realized.

Furthermore, since the movements displayed as hints are the lateralinversions of the assigned movements, the player can intuitivelydetermine the correct movements by visually viewing the hints and movereflectively. Accordingly, the time period from the display of a hint tothe actual performance by the player may be minimized.

Furthermore, since the player performs the movements in synchronizationwith the tempo of the music, the voice, the effect sounds or the videoetc., amusement of performing the movements can further be enhanced.

Furthermore, since the processes for the feature extraction etc. todetect the movements of the player can be performed on the side of theartificial retina chip, the processing speed of the apparatus as a wholemay be accelerated.

The entire disclosures of Japanese Patent Application Nos. Tokugan2000-42048 filed on Feb. 18, 2000, Tokugan 2000-42049 filed on Feb. 18,2000, and Tokugan 2000-42052 filed on Feb. 18, 2000, includingspecifications, claims, drawings and summaries, are incorporated hereinby reference in its entirety.

1. A game apparatus for executing a predetermined game, comprising: animage sensor that is capable of concurrently detecting movements of aplayer in a detection range for a single view; a movement detection unitthat detects a movement of the player in each of a plurality ofdetection regions from a detection result of the image sensor, thedetection range for the single view being divided to form the pluralityof detection regions; a storage unit that stores one or morepredetermined assigned movements for each of the plurality of detectionregions; a similarity decision unit that determines a similarity betweenthe movements of the player and the predetermined assigned movements foreach detection region based on at least one of a direction, a magnitudeand a speed of the movements of the player; and a game level settingunit that sets a game level of difficulty, wherein at least one of atempo of the predetermined assigned movements, a type of thepredetermined assigned movements and a degree of similarity is changedbased on the game level of difficulty.
 2. A game apparatus according toclaim 1, wherein each of the predetermined assigned movements aredefined for a corresponding predetermined time, and the similaritydecision unit determines the similarity between the movements of theplayer and each of the predetermined assigned movements for thecorresponding predetermined time.
 3. A game apparatus according to claim1, further comprising: a timing notice unit that indicates apredetermined timing by a visual effect or an auditory effect; and atiming decision unit that decides whether a timing of the movements ofthe player for each detection region detected by the movement detectionunit correspond to the predetermined timing.
 4. A game apparatusaccording to claim 1, further comprising: a movement specifying unitthat specifies, among movements of the player for each of the detectionregions, zero or one movement of the player for each of the detectionregions detected by the movement detection unit; wherein the similaritydecision unit decides the similarity between the movement of the playerspecified by the movement specifying unit and the predetermined assignedmovement for each of the detection regions.
 5. A game apparatusaccording to claim 4, wherein a plurality of concurrent movements of theplayer is detected in one detection region by the movement detectionunit, and the movement specifying unit specifies one of the concurrentmovements of the player detected in the one detection region based on atleast one of the direction, the magnitude and the speed of the movement.6. A game apparatus according to claim 1, wherein a boundary between theplurality of detection regions is changed according to a predeterminedcondition detected by the movement detection unit.
 7. A game apparatusaccording to claim 1, wherein the movement detection unit comprises anartificial retina chip for detecting the movement of the player.
 8. Agame apparatus according to claim 4, wherein the movement detection unitdetects a detected movement of the player that extends over more thanone of the detection regions, and the similarity decision unitdetermines the similarity between the detected movement of the playerand the predetermined assigned movement in each of the detectionregions.
 9. A game apparatus according to claim 1, further comprising aby-region decision display unit that displays a decision according to asimilarity in each detection region.
 10. A game apparatus according toclaim 1, wherein the similarity decision unit changes a degree ofsimilarity based on a predetermined condition detected by the movementdetection unit.
 11. A game apparatus according to claim 6, wherein thepredetermined condition relates to a body shape of the player detectedby the movement detection unit.
 12. A game apparatus according to claim10, wherein the predetermined condition relates to a body shape of theplayer detected by the movement detection unit.
 13. A game apparatus forexecuting a predetermined game, comprising: a storage unit that storesone or more predetermined assigned movements, the predetermined assignedmovements having a predetermined timing; a movement detection unit thatdetects one or more concurrent movements of a player; a timing noticeunit that indicates the predetermined timing of the predeterminedassigned movements by a visual effect or an auditory effect; a timingdecision unit that decides whether a timing of the concurrent movementsof the player corresponds to the predetermined timing for thepredetermined assigned movements and decides that the timing of theconcurrent movement of the player does not correspond to thepredetermined timing for the predetermined assigned movements if thetiming of the concurrent movements of the player does not fall within anallowable range of the predetermined timing for the predeterminedassigned movements; a movement specifying unit that specifies acorrespondence between the concurrent movements of the player and thepredetermined assigned movements based on the decision by the timingdecision unit; a similarity decision unit that determines a similaritybetween the concurrent movements of the player and the predeterminedassigned movement based on at least one of a direction, a magnitude anda speed of the concurrent movements of the player; and a game levelsetting unit that sets a game level of difficulty, wherein at least oneof a tempo of the predetermined assigned movements, a type of thepredetermined assigned movements and a degree of similarity is changedbased on the game level of difficulty.
 14. A game apparatus according toclaim 13, further comprising a detection range of the movement detectionunit, the detection range being divided into a plurality of detectionregions, wherein the similarity decision unit determines the similaritybetween the concurrent movements of the player and the predeterminedassigned movements for each detection region.
 15. A storage medium in acomputer having a computer-executable program recorded thereon, thecomputer being provided with an image sensor that is capable ofconcurrently detecting movements of a player in a detection range for asingle view, the program comprising: a program code that executes apredetermined game; a program code that detects a movement of the playerin each of a plurality of detection regions from a detection result ofthe image sensor, the detection range for the single view being dividedto form the plurality of detection regions; a program code thatdetermines a similarity between the movement of the player and apredetermined assigned movement for each detection region based on atleast one of a direction, a magnitude and a speed of the movement of theplayer, the predetermined assigned movement being set in accordance withthe detection region in advance; and a program code that sets a gamelevel of difficulty, wherein at least one of a tempo of thepredetermined assigned movement, a type of the predetermined assignedmovement and a degree of similarity is changed based on the game levelof difficulty.
 16. A storage medium having a computer program forperforming the steps of: executing a predetermined game; detecting amovement of a player in each of a plurality of detection regions from adetection result of an image sensor that is capable of concurrentlydetecting movements of the player in a detection range for a singleview, the detection range for the single view being divided to form theplurality of detection regions; determining a similarity between themovement of the player and a predetermined assigned movement for eachdetection region based on at least one of a direction, a magnitude and aspeed of the movement of the player, the predetermined assigned movementbeing set in accordance with the detection region in advance; andsetting a game level of difficulty, wherein at least one of a tempo ofthe predetermined assigned movement, a type of the predeterminedassigned movement and a degree of similarity is changed based on thegame level of difficulty.
 17. A game apparatus for executing apredetermined game, comprising: a storage unit that stores one or morepredetermined assigned movements; an image sensor that is capable ofconcurrently detecting movements of a player in a detection range for asingle view; a movement detection unit that detects a movement of theplayer in each of a plurality of detection regions from a detectionresult of the image sensor, the detection range for the single viewbeing divided to form the plurality of detection regions; a movementspecifying unit that specifies one or more specified detection regionsfor the concurrent movements of the player based on at least one of adirection, a magnitude and a speed of each of the movements of theplayer; a similarity decision unit that determines a similarity in eachspecified detection region between the concurrent movements of theplayer and the predetermined assigned movements corresponding to thespecified detection region based on at least one of the direction, themagnitude and the speed of the concurrent movement of the player; and agame level setting unit that sets a game level of difficulty, wherein atleast one of a tempo of the predetermined assigned movements, a type ofthe predetermined assigned movements and a degree of similarity ischanged based on the game level of difficulty.
 18. A game apparatus forexecuting a predetermined game, comprising: a storage unit that storesone or more first movement vectors of a predetermined assigned movement;a movement detection unit that detects a movement of a player; amovement vector calculation unit that calculates one or more secondmovement vectors for the movement of the player based on at least one ofa direction, a magnitude and a speed of the movement of the player; anda similarity decision unit that decides a similarity between the firstmovement vectors and the second movement vectors.